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I. 


INTRODUCTION 


A.  BACKGROUND 

During  a  time  period  when  the  discussion  of  defense  budget  cuts  and  sequestration 
dominates  the  pages  of  national  and  defense  news,  what  strategy  is  the  military  employing  to 
operate  more  efficiently?  The  wars  in  Iraq  and  Afghanistan  are  coming  to  a  close,  and  with 
that,  a  further  reduction  in  operating  budgets  is  expected  across  the  Department  of  Defense 
(DoD).  The  need  for  a  prudent  and  accountable  expenditure  of  taxpayer  dollars  has  reached 
its  pinnacle  with  the  requirement  for  the  DoD  to  be  100%  auditable  by  2017,  per  Secretary  of 
Defense  Panetta’s  direction.  The  priority  of  fiscal  responsibility  has  gone  as  far  as  to  be 
embedded  in  the  National  Security  Strategy,  as  stated  in  the  following  excerpt: 

Cost-effective  and  efficient  processes  are  particularly  important  for  the 
Department  of  Defense,  which  accounts  for  approximately  70  percent  of  all 
Federal  procurement  spending.  We  will  scrutinize  our  programs  and 
terminate  or  restructure  those  that  are  outdated,  duplicative,  ineffective,  or 
wasteful.  The  result  will  be  more  relevant,  capable,  and  effective  programs 
and  systems  that  our  military  wants  and  needs.  (President  of  the  United  States, 

2010,  p.  34) 

In  addition  to  spending  funds  more  wisely,  further  motivation  behind  this  research  is 
to  improve  the  overall  condition  of  readiness  experienced  across  the  Navy’s  fleet.  Readiness 
is  measured  in  various  ways,  ranging  from  detailed  ship-wide  assessments  to  the  submission 
of  an  individual  casualty  report  (CASREP).  A  CASREP  is  a  report  made  to  the  ship’s 
operational  chain  of  command  concerning  a  significant  equipment  malfunction  that  cannot  be 
corrected  within  a  48-hour  period.  Due  to  the  frequency  and  specific  nature  of  CASREPs, 
these  reports  played  a  significant  role  in  the  analysis  discussed  in  this  paper. 

In  a  2012  report,  the  Government  Accountability  Office  (GAO)  provided  evidence, 
with  results  displayed  in  Figure  1,  concerning  a  noticeable  increase  in  the  number  of 
CASREPs  reported  from  2008  to  2012;  as  the  GAO  (2012)  noted,  an  increase  in  the  number 
of  CASREPs  would  indicate  a  decline  in  overall  material  readiness.  In  our  research,  we  used 
CASREP  frequency  and  severity  as  a  measure  of  effectiveness,  and  most  important,  we 
considered  only  those  CASREPs  that  require  additional  repair  parts  due  to  onboard 
allowance  shortages. 
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Figure  1.  Average  Number  of  Casualty  Reports  of  Surface  Combatant  and 
Amphibious  Warfare  Ships  by  Quarter  From  January  2008  Through  March  2012 

(GAO,  2012) 

For  the  U.S.  military  to  remain  effective  as  funding  is  reduced,  efficiencies  and  cost¬ 
saving  opportunities  must  be  identified  and  implemented  to  ensure  that  the  nation  can  sustain 
a  competitive  advantage  in  the  21st  century.  There  are  myriad  ways  that  the  DoD  can 
achieve  cost  savings  from  the  numerous  programs  in  development,  as  well  as  from  those  that 
have  reached  maturity  and  are  currently  in  the  sustainment  phase.  In  this  research,  we  seek 
to  identify  one  such  avenue  for  the  U.S.  Navy  that  would  require  a  minimal  investment  and 
result  in  maximum  effectiveness. 

In  addition  to  the  fact  that  the  DoD  has  been  mandated  to  reduce  spending  and  find 
efficiencies  wherever  possible,  other  significant  reasons  justify  why  such  research  should 
take  place.  Following  the  events  of  September  11,  the  entire  DoD  saw  a  dramatic  increase  in 
the  amount  of  funding  appropriated  for  operations  and  maintenance  (O&M;  Office  of 
Management  and  Budget  [OMB],  2003).  This  O&M  funding  is  used  to  pay  for  the  day-to- 
day  activities  of  our  deployed  or  deployable  forces  and  typically  consists  of  the  costs 
associated  with  fuel,  repair  parts,  and  maintenance.  The  increase  in  O&M  funding  resulted 
in  a  decrease  in  acquisition  funding  that  covers  the  research,  development,  and  production  of 
the  next  generation  of  weapons  systems.  As  the  campaigns  in  Afghanistan  and  Iraq  expire, 
O&M  funding  is  expected  to  decrease  and  return  to  more  traditional,  peacetime  levels. 
However,  the  expected  decrease  in  O&M  funding  does  not  guarantee  a  subsequent  increase 
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in  procurement  funding.  This  distinction  is  important  to  note,  because  the  findings  of  the 
present  research  may  impact  both  appropriation  types  and,  if  successful,  would  result  in  a 
reduction  of  total  ownership  cost  (TOC). 

According  to  the  Defense  Acquisition  University  (DAU,  2012),  TOC  is  defined  as  a 
concept  designed  to  detennine  the  true  cost  of  the  design,  development,  ownership,  and 
support  of  DoD  weapons  systems.  Within  TOC  are  the  acquisition  costs  and  operations  and 
sustainment  (O&S)  costs  of  the  system,  with  the  O&S  costs  accounting  for  a  significantly 
larger  segment.  Figure  2  presents  the  breakdown  of  TOC  of  a  weapons  system  with  greater 
clarity. 


Figure  2.  Total  Ownership  Cost  Throughout  the  System  Life  Cycle 
(Defense  Systems  Management  College,  1997,  p.  181) 

Figure  2  makes  clear,  and  the  GAO  has  agreed,  that  significantly  more  cost  savings 
can  be  generated  throughout  the  O&S  phase  as  a  result  of  more  aggressive  research  during 
the  procurement  phase  of  the  system  life  cycle  (GAO,  2010).  A  reduction  in  O&S  costs 
translates  directly  to  a  reduction  in  the  amount  of  O&M  funding  required  for  agencies  to 
operate,  which  further  results  in  an  overall  savings  by  the  DoD.  As  mentioned  previously,  a 
reduction  in  O&M  funding  would  provide  the  opportunity  to  invest  more  in  the  acquisition  of 
critical  programs  and  potentially  improve  our  national  defense  capabilities.  Our  research 
focuses  on  the  O&S  phase,  because  this  is  the  phase  that  requires  the  largest  amount  of 
funding. 
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B. 


SCOPE 


The  scope  of  this  project  is  to  conduct  a  business  case  analysis  (BCA)  of  the 
Continuous  Integrated  Logistics  Support-Targeted  Allowancing  Technique  (CILS— TAT) 
with  the  assistance  of  the  Naval  Sea  Logistics  Center  (NSLC).  According  to  the  NSLC 
(2012)  command  description,  “NSLC  is  a  field  activity  of  the  Naval  Undersea  Warfare 
Center,  and  is  tasked  with  providing  integrated  logistics,  engineering,  and  information 
technology  expertise  to  all  facets  of  the  Navy's  worldwide  logistics  support  structure.” 

Embedded  in  NSLC’s  integrated  logistics  support  responsibilities  are  the 
requirements  to  manage  the  configuration  of  naval  weapons  systems  within  the  different 
classes  of  ships  and  submarines,  as  well  as  to  provide  the  appropriate  allowances  for  spare 
parts  to  be  stocked  onboard.  Configuration  management,  while  closely  related  to  the 
allowancing  process,  is  beyond  the  scope  of  our  research  and  therefore  is  not  fully  introduced 
here.  Our  research  instead  focuses  on  a  subset  of  the  allowancing  process  known  as  CILS- 
TAT.  We  discuss  this  technique,  as  well  as  the  traditional  allowancing  process,  in  much 
greater  detail  in  subsequent  chapters  and  provide  a  better  understanding  of  the  variables 
considered  when  NSLC  is  determining  allowances. 

Our  current  motivation  behind  evaluating  the  effectiveness  of  the  CILS-TAT  is  the 
suspension  of  the  program  since  2009,  when  the  U.S.  Navy  began  its  transition  to  an 
enterprise  resource  planning  (ERP)  framework.  We  expect  that  by  the  time  this  research  is 
published,  the  U.S.  Navy’s  allowancing  process  systems  will  have  fully  transitioned  into  the 
ERP  framework;  therefore,  the  time  for  a  possible  re-implementation  of  the  CILS-TAT  is 
quickly  approaching.  Based  on  our  findings,  we  hope  to  demonstrate  the  level  of 
effectiveness  achieved  as  a  result  of  using  the  CILS-TAT  and  provide  recommendations 
concerning  the  use  of  the  program  in  the  future. 

The  CILS-TAT  can  be  used  onboard  any  U.S.  Navy  platform.  To  further  narrow  the 
scope  of  our  project,  we  focused  on  18  ships  from  the  U.S.  Navy’s  Ballistic  Missile  Defense 
(BMD)  Fleet.  Five  Ticonderoga-class  guided  missile  cruisers  (CGs)  and  13  Arleigh  Burke- 
class  guided  missile  destroyers  (DDGs)  make  up  our  sample.  Critical  to  the  U.S.  National 
Security  Strategy,  the  mission  of  the  BMD  Fleet  is  to  detect,  track,  and  intercept  ballistic 
missiles  of  all  ranges  and  types  (Missile  Defense  Agency,  n.d.).  The  principal  reason  for  our 
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selection  of  these  ships  is  not  only  the  importance  of  the  BMD  mission  but  also  the 
commonality  of  systems  contained  within  the  Ticonderoga  and  Arleigh  Burke  classes.  With 
the  Arleigh  Burke  class  still  in  production,  the  results  of  this  research  will  provide  the  longest 
period  of  applicability  and  support  the  largest  class  of  ships  in  the  naval  inventory. 

C.  PROBLEM  DESCRIPTION 

The  task  of  minimizing  costs  while  maximizing  effectiveness  is  not  one  that  can 
easily  be  achieved  and,  in  some  cases,  may  not  be  realistic.  The  goal  of  this  project, 
however,  is  to  do  just  that  through  an  aggressive  spare -part  allowancing  model  that  will 
improve  the  effectiveness  of  the  U.S.  Navy  surface  fleet  through  a  reduction  in  O&S  costs. 
The  project  analysis  covers  two  distinct  time  periods  to  capture  the  true  value  of  the  CILS- 
TAT  and  then  uses  forecasting  models  to  determine  the  future  viability  of  the  program.  The 
first  period  covers  the  years  2003-2009.  During  this  time,  18  CILS-TAT  reviews  were 
conducted  and  48  months  of  operational  data  were  available  per  ship.  During  the  second 
period,  from  2009  to  2011,  no  CILS-TATs  were  conducted,  and  no  other  allowancing 
maintenance  was  done.  Based  on  the  data  available  for  analysis  during  these  periods,  we 
attempted  to  forecast  the  long-term  effectiveness  of  the  CILS-TAT. 

Unlike  land-based  forces,  where  the  addition  of  another  warehouse  to  store  repair 
parts  is  not  a  major  concern,  on  a  U.S.  Navy  ship,  there  is  a  finite  amount  of  space  to  store 
repair  parts.  In  a  perfect  world,  a  ship  could  carry  a  replacement  part  for  every  installed 
component  in  the  event  that  a  repair  is  required.  Because  that  is  not  feasible,  a  great  deal  of 
consideration  must  be  given  to  determine  the  right  mix  of  parts  that  make  up  the  ship’s 
onboard  allowance  list.  A  shipboard  allowance  list  for  maintenance  parts  is  the  larger  list 
from  which  spare  items  are  selected,  or  not  selected,  for  onboard  allowances. 

In  the  past,  one  problem  with  such  lists  has  been  assuring  maximum  value  received 
for  dollars  spent  on  spare  items.  Such  value  may  be  received  only  by  making  sure  that  spare 
items  are  ordered  through  some  type  of  a  combinatorial  optimization  process  (Harrahy, 
Powell,  &  Lutz,  1968).  According  to  the  GAO  (2003),  the  U.S.  Navy’s  spare-parts  supply 
problems  can  delay  the  completion  of  needed  maintenance  and  repair  jobs  on  deployed  ships 
and  can  affect  their  operations  and  mission  readiness.  In  the  same  2003  GAO  report, 
covering  two  carrier  strike  groups  over  six  deployments,  58%  of  the  50,000  maintenance 
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work  requests  were  delayed  because  the  appropriate  parts  were  not  located  on  the  ships.  It 
can  be  deduced  from  these  studies  that  getting  the  right  mix  of  parts  onboard  a  ship  is  a 
costly,  complex,  and  critical  task  with  far-reaching  implications.  The  processes  involved  in 
allowancing  have  evolved  over  time.  We  cover  two  such  processes  in  subsequent  chapters. 

Over  the  course  of  several  decades,  initiatives  have  been  implemented  to  either 
improve  the  business  practices  associated  with  allowancing  or  reduce  slow-moving  and 
unnecessary  inventory  (K.  R.  Bitner,  personal  communication,  November  5,  2012). 
Allowances  can  be  reduced  for  a  number  of  reasons  ranging  from  obsolescence  to  a  lack  of 
demand  from  the  system.  When  an  allowance  is  reduced  due  to  a  lack  of  demand,  greater 
risk  is  incurred  in  the  event  of  an  equipment  casualty.  The  severity  of  a  casualty  can  range 
from  an  insignificant  impact  to  the  loss  of  a  critical  system  and  failure  to  complete  a  primary 
mission.  These  circumstances  obviously  cover  a  wide  spectrum  but  hopefully  provide  some 
insight  into  the  factors  that  must  be  considered  at  the  component  level  when  determining 
allowances. 

One  proposal  to  help  the  U.S.  Navy  more  efficiently  use  its  appropriated  funds  has 
been  to  spend  less  in  the  procurement  of  spare-parts  allowances  onboard  ships.  The  problem 
with  reducing  the  number  of  parts  onboard  is  that  when  a  system  fails  and  the  parts  are  not 
onboard,  logistics  support  is  required — whether  the  part  is  coming  from  the  other  side  of  the 
globe  or  from  a  warehouse  a  few  miles  away.  In  addition  to  the  required  logistics  support,  the 
system  requiring  the  part  is  either  non-operational  or  degraded  while  awaiting  repairs.  The 
costs  of  the  part  not  being  onboard  can  be  measured  quantitatively  in  monetary  terms  and 
more  abstractly  by  a  decrease  in  the  ship’s  capabilities. 

The  monetary  costs  of  providing  material  support  to  an  operational  vessel  are  the 
most  simple  to  compute  and  typically  involve  determining  the  cost  of  the  part  required,  the 
location  of  the  vessel  requiring  it,  and  the  urgency  of  the  need  for  the  item.  Transportation 
costs  are  reasonably  easy  to  obtain  and  can  be  forecasted  for  a  variety  of  scenarios  using 
simulations.  The  more  difficult  cost  to  capture  is  the  significance  of  losing  a  mission-critical 
system  for  an  extended  period  as  a  result  of  not  having  the  parts  in  the  right  place  at  the  right 
time. 
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RESEARCH  OBJECTIVES 


The  objectives  for  our  research  include  the  following: 

1 .  Conduct  a  BCA  of  the  CILS-TAT  process  using  historical  data. 

2.  Measure  the  costs  associated  with  performing  a  CILS-TAT. 

3.  Provide  recommendations  to  NSLC  concerning  possible  improvements  to  the 
existing  CILS-TAT  model. 

RESEARCH  QUESTIONS 

We  have  addressed  the  following  questions  in  our  research: 

1.  How  did  CILS-TAT  impact  allowance  effectiveness  for  our  sample,  and  at 
what  cost? 

2.  Are  there  correlations  between  allowance  effectiveness  rates,  percentage  of 
time  free  (POTF)  from  CASREPs,  and  CILS-TAT  cost? 

3.  What  was  the  effect  of  not  having  CILS-TAT  during  2009-2011,  when 
traditional  allowancing  procedures  were  not  available? 
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BACKGROUND 


F.  ALLOWANCING  FUNDAMENTALS 

When  considering  the  austere  environment  within  which  U.S.  Navy  ships  operate  and 
the  absolute  necessity  to  function  above  readiness  metrics,  it  is  important  that  the  U.S.  Navy 
develop  and  implement  a  robust  set  of  allowancing  procedures  to  ensure  that  the  right  parts 
are  on  the  shelf  when  equipment  fails.  As  related  to  inventory  management,  allowancing  can 
be  defined  as  determining  the  correct  blend  of  items  carried  onboard  to  ensure  that  the  part  is 
available  to  restore  the  equipment  when  failures  occur.  The  correct  quantity  of  items  is 
influenced  by  constraints  such  as  funding  and  the  cost  of  procurement,  the  availability  of  the 
item,  space  availability  onboard  the  ship,  mean  time  between  failures  (MTBFs),  and 
estimated  lead-times  for  replenishment. 

As  mentioned  previously,  the  entire  purpose  of  the  allowancing  process  is  to  generate 
the  right  set  of  parts  to  stock  onboard  the  ship,  thereby  ensuring  that  the  overall  mission 
readiness  levels  of  the  vessel  do  not  drop  below  prescribed  metrics  set  forth  by  the  U.S. 
Navy.  To  measure  the  effectiveness  of  the  allowancing  process,  the  U.S.  Navy  has 
formulated  several  perfonnance  metrics,  such  as  operational  availability  (A0),  supply  gross 
issue  effectiveness,  supply  net  issue  effectiveness,  CASREP  frequency,  and  POTF  from 
CASREPs. 

1.  Operational  Availability 

A0  provides  a  method  of  predicting  and  assessing  system  performance  and  readiness 
during  the  acquisition  process  and  then  becomes  the  performance  benchmark  during  initial 
operational  capability  (IOC),  deployment,  and  operations/maintenance  cycles  (Chief  of 
Naval  Operations  [CNO],  2003).  The  U.S.  Navy’s  ability  to  meet  the  highest  readiness 
levels  is  principally  derived  by  the  A0  of  the  warfighting  systems  and  equipment  installed 
onboard  U.S.  Navy  vessels. 

According  to  Chief  of  Naval  Operations  Instruction  (OPNAVINST)  3000. 12A  (CNO, 
2003),  the  calculation  for  determining  the  A0  is  a  probability  function  of  reliability, 
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maintainability,  and  supportability  components.  The  equation  for  determining  A0  is  written 
as  the  following:  system  up  time  divided  by  total  time  (up  time  +  down  time). 

A0  is  fundamentally  considered  as  a  supportability  calculation  of  the 
equipment/system  in  terms  of  predicted  reliability  (R),  called  mean  time  between  failure 
(MTBF);  predicted  maintainability  (M),  in  terms  of  mean  time  to  repair  (MTTR);  and 
designed  supportability,  called  mean  logistics  delay  time  (MLDT;  CNO,  2003).  Inherent 
availability  (A;),  an  alternative  metric,  does  not  include  a  consideration  of  support  functions 
related  to  re-supply  transportation  and  repair.  Once  A;  is  determined,  one  is  ready  to  add  in 
the  supportability  portion,  or  MLDT,  to  calculate  A0  (CNO,  2003).  See  Figure  3  for  a 
graphical  representation  of  Ai  and  A0. 


A;  = 

MTBF 

MTBF  +  MTTR 

MTBF 

A0  = 

MTBF  +  MTTR 

+  MLDT 

Hardware/Software 
Design  Considerations 

Logistics  System 
Design  Considerations 

Figure  3.  Logistics  Impact  on  Operational  Availability 
(CNO,  2003) 


2.  Measuring  Allowancing  Effectiveness 

In  order  to  effectively  measure  the  allowancing  packages  that  the  U.S.  Navy  is 
developing  and  sending  to  the  fleet,  the  U.S.  Navy  has  constructed  several  calculations  to 
interpret  the  demand  signal  against  the  assets  onboard.  The  metrics  are  assembled  by 
segregating  the  demand  into  different  pools  or  supply  source  codes  based  on  the  stock 
posture  of  the  requirement.  Figure  4  represents  the  various  codes. 
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CODE  DEFINITION 

A  Allowance  List  Material  issued  from  storeroom  stock 
C  Non- Allowance  List  Material  (SIM)  issued  from  storeroom  stock 

D  Allowance  List  Material  Not  In  Stock  (NIS)  when  requested 
Non- Allowance  List  Material  (SIM)  Not  In  Stock  (NIS)  when 
requested 

Not  Carried  (NC)  repair  parts  which  are  not  listed  on  an  APL  in  the 
ship's  COS  file 

Not  Carried  (NC)  repair  parts  which  are  listed  on  an  APL,  but 
does  not  compute  lor  an  allowance 

Figure  4.  U.S,  Navy  Allowance  Supply  Source  Codes 
(Naval  Supply  Systems  Command  [NAVSUP],  1997) 

The  calculation  used  in  evaluating  allowance  performance  is  the  allowance 
effectiveness  metric,  which  is  represented  by  the  following  equation:  (A  +  C  +  D  +  F) / (A  + 
C  +  D  +  F  +  G+  J ).  The  allowance  effectiveness  calculation  allows  stakeholders  to 
determine  the  strength  of  the  allowanced  items,  as  well  as  the  non-allowance  material 
selective  item  maintenance  (SIM)  against  demand  (NAVSUP,  1997). 

3.  Supply  Issue  Effectiveness  Rates 

In  an  effort  to  measure  the  overall  issue  effectiveness  of  the  allowancing  packages 
throughout  the  training  cycle  and  into  deployment,  the  U.S.  Navy  has  developed  a  set  of 
inventory  issue  effectiveness  rates  to  judge  the  ability  of  the  inventory  to  meet  the  demands 
placed  against  that  inventory.  There  are  two  metrics  that  are  used  to  determine  the  success  of 
the  allowancing  package,  the  first  being  the  supply  gross  issue  effectiveness  rate.  Supply 
gross  issue  effectiveness  measures  the  number  of  requirements,  commonly  called 
requisitions,  issued  against  the  total  number  of  requirements.  Supply  gross  issue 
effectiveness  is  computed  by  dividing  the  number  of  requisitions  issued  by  the  total  number 
of  requisitions:  (A  +  C)/ (A  +  C  +  D  +  F  +  G  +  J). 

The  aforementioned  performance  measurement  allows  the  managers  of  the 
allowancing  package  to  quantitatively  judge  whether  or  not  they  have  the  correct  variety  and 
quantity  of  parts  on  the  ship.  The  calculation  provides  a  percentage  value  for  every  demand 
that  was  issued  from  the  storeroom.  In  other  words,  it  reveals  how  many  parts  were  not 
onboard  that  should  have  been  onboard  given  the  current  demand.  The  output  from  this 
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calculation  would  provide  an  allowancing  package  stakeholder  with  a  sense  of  how  many 
parts  or  line  items  must  be  added  to  the  allowancing  package  to  meet  issue  effectiveness 
goals. 

The  second  metric  that  the  U.S.  Navy  uses  is  the  supply  net  issue  effectiveness  rate. 
This  measurement  tool  is  calculated  by  dividing  the  total  number  of  issued  requisitions  by  the 
total  number  of  requisitions  minus  the  number  of  requisitions  not  carried  (NC)  onboard: 
(A  +  C) /{A  +  C  +  D  +  F) .  This  more  focused  rate  allows  the  stakeholders  to  obtain  a  sense 
of  how  the  variety  and  quantity  of  the  parts  included  in  the  allowancing  package  are  reacting 
to  the  current  demand  signal.  For  the  purpose  of  allowancing  package  development,  the  use 
of  the  supply  gross  issue  effectiveness  rate  would  prove  to  be  more  useful  because 
stakeholders  desire  to  know  on  average  how  well  the  allowancing  package  supported  the 
demand  as  a  whole. 

4.  Casualty  Reporting 

A  measure  of  effectiveness  central  to  our  project  involves  tracking  the  number  and 
severity  of  CASREPs  for  each  of  the  18  ships  in  the  sample.  A  CASREP  is  a  report  made  to 
the  ship’s  operational  chain  of  command  concerning  a  significant  equipment  malfunction  that 
cannot  be  corrected  within  a  48-hour  period.  The  release  of  a  CASREP  will  also  alert 
supporting  activities  of  the  urgency  to  provide  aid  in  the  form  of  spare  parts  or  technical 
assistance.  The  significance  of  each  CASREP  is  determined  by  the  impact  of  the  equipment 
failure  on  the  ship’s  mission.  There  are  different  categories  of  CASREPs,  ranging  from 
Category  Two  (C2)  through  Category  Four  (C4),  with  C4  being  the  most  severe.  A  C4 
CASREP  denotes  that  a  deficiency  exists  in  mission-essential  equipment  that  causes  the  loss 
of  at  least  one  primary  mission.  A  C3  CASREP  denotes  that  a  deficiency  exists  in  mission- 
essential  equipment  that  causes  a  major  degradation  but  not  the  loss  of  a  primary  mission. 
Finally,  a  C2  CASREP  indicates  that  the  ship  has  lost  redundancy  in  one  of  its  primary  or 
secondary  mission  areas.  CASREPs  are  required  to  be  updated  periodically  by  the  ship  until 
the  damaged  equipment  is  restored  to  a  fully  mission-capable  state.  The  frequency  of 
updates  is  correlated  to  the  severity  of  the  CASREP.  The  CASREP  will  remain  an  open 
report  until  all  repairs  are  finalized  (NAVSUP,  1997). 
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5. 


Percentage  of  Time  Free  From  Casualty  Reports 


POTF  is  a  readiness  metric  tracked  by  NAVSUP  Corporate  Information  Systems 
(CIS)  that  records  the  number  of  days  in  a  reporting  period  that  a  ship  does  not  have  an  open 
C3  or  C4  CASREP.  This  metric  is  a  central  measure  of  a  ship’s  material  readiness  and  one 
that  we  use  in  the  analysis  section  of  this  report.  A  calculation  of  time  free  of  casualties  is 

K  /  (days  out  of  shipyard  and  free  of^CASREPS  during  period)/ 

^a=1l  (Calendar  days  during  period)  I 

100  x - - - ^  (1) 

where 

i  =  a  running  index  for  individual  ships  in  the  grouping  that  have  been  active 
for  more  than  1/5  of  the  calendar  period,  and 
K  =  total  ships  in  the  grouping  meeting  the  active  time  criteria. 

G.  ALLOWANCING  STAKEHOLDERS 

The  entire  allowancing  process  is  completed  by  a  vast  array  of  logistics  and 
maintenance  professionals  employed  by  the  U.S.  Navy  and  civilian  partners.  The 
requirements  necessary  to  develop  a  comprehensive  allowancing  package  deem  that  all 
influencers  collaborate  effectively  as  one  cohesive  team.  The  requirements  for  the 
allowancing  package  range  from  ship  configuration  data,  maintenance  and  logistics  data,  and, 
if  available,  logistics  support  data  specifically  related  to  reliability  concerns. 

On  the  logistics  side  of  the  house,  there  are  a  couple  of  key  players  across  the  U.S. 
Navy.  The  first  of  these  players — and  probably  the  most  dominant  in  calculating  the 
allowances — is  the  NAVSUP  Weapons  System  Support,  Mechanicsburg  (WSS  Mech).  The 
WSS  Mech  is  the  U.S.  Navy’s  representative  to  manage  the  allocation  of  resources  relating  to 
the  allowancing  process.  NAVSUP  WSS  Mech  possesses  the  models  necessary  to  develop 
allowancing  packages  and  work  with  all  stakeholders  to  ensure  that  the  end  product  fits  the 
needs  of  the  warfighter.  In  addition,  it  is  tasked  with  controlling  the  funding  for  initial 
outfitting  onboard  the  vessels. 

On  the  maintenance  side  of  the  house,  the  stakeholder  most  concerned  with  the 
development  of  the  allowancing  process  is  the  Naval  Sea  Systems  Command  (NAVSEA). 
NAVSEA  is  tasked  with  ensuring  that  the  ship  is  configured  properly  with  the  most  up-to- 
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date  systems  and  equipment.  To  ensure  that  the  configuration  of  all  systems  is  correct, 
NAVSEA  works  closely  with  NSLC,  which  provides  integrated  logistics  support.  The 
configuration  of  the  ship  must  be  accurate  when  developing  the  allowancing  package  to 
facilitate  superior  sustainability  during  the  entire  life  cycle  of  the  ship. 

When  developing  a  product,  there  must  be  a  customer.  In  this  case,  the  customer  is 
the  warfighter  on  the  waterfront:  Commander,  Fleet  Forces  Command  (CFFC).  CFFC  is 
charged  by  the  CNO  to  ensure  that  all  fleet  forces  maintain  the  highest  levels  of  operational 
readiness  to  support  the  nation’s  maritime  strategy.  With  regard  to  surface  ships, 
Commander,  Naval  Surface  Forces  (SURFOR)  is  the  stakeholder  involved  in  the  allowancing 
process  and  subsequent  reviews  of  that  process  as  the  ship  progresses  throughout  its  life 
cycle.  Within  SURFOR,  there  are  maintenance  and  logistics  components  that  work  in 
concert  with  their  counterparts  involved  in  the  process.  Fundamentally,  once  SURFOR  takes 
ownership  of  the  vessel,  it  is  ultimately  responsible  if  the  ship  does  not  meet  the  mission. 

The  maintenance  personnel  at  SURFOR  manage  the  maintenance  practices  and 
configuration  changes  related  to  keeping  the  ship  up  to  date.  The  logisticians  are  constantly 
monitoring  the  issue  effectiveness  rates  to  determine  whether  an  allowancing  package  should 
be  applied  to  increase  the  mission  readiness. 

H.  FUNDING 

The  allowancing  process  requires  two  different  classifications  of  funding  to  pay  for 
all  inventory  required:  initial  issue  provisioning  and  follow-on  replenishment  over  the  life 
cycle  of  the  allowance  being  tied  to  that  vessel.  The  reasoning  behind  using  two  different 
classifications  of  money  is  the  delineation  between  the  budget  activities  (BA)  of  Operations 
and  Maintenance,  Navy  (0&M,N)  and  Other  Procurement,  Navy  (OPN)  within  the  DoD 
budget.  The  two  classifications  of  money  are  controlled  by  separate  organizations  within  the 
U.S.  Navy. 

The  funding  stream  more  closely  tied  to  the  allowancing  process  and  initial 
provisioning  is  the  OPN  appropriation.  This  appropriation  is  designated  for  the  procurement, 
production,  and  modernization  of  support  equipment  and  materials  not  otherwise  provided 
for,  as  well  as  the  procurement  and  installation  of  equipment  (Department  of  the  Navy 
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[DoN],  2012).  Because  of  the  designation  of  these  funds  to  support  the  initial  outfitting  of 
naval  forces,  all  initial  provisioning  within  an  allowancing  package  are  paid  for  with  OPN 
funds. 

The  allocation  of  these  funds  from  Congress  is  given  to  NAVSUP  WSS  Mech  to 
determine  the  best  allocation  across  the  competing  priorities  within  the  enterprise.  The  funds 
are  then  designated  as  OPN-8  funds.  The  Navy  Working  Capital  Fund  (NWCF)  then  buys 
spare  parts  listed  in  the  new  allowancing  product  with  OPN-8  funds  following  the  post¬ 
material  support  date  (MSD;  CNO,  2012).  Once  the  allowancing  products  are  developed 
with  the  inputs  from  stakeholders,  NAVSUP  WSS  Mech  makes  the  determination  to 
purchase  the  requisite  spare  parts  to  fill  the  allowances. 

The  appropriations  assigned  to  the  BA  of  0&M,N  are  designated  to  finance  the  day- 
to-day  costs  of  operating  naval  forces,  including  fuel,  supplies,  and  maintenance  of  ships; 
U.S.  Navy  and  Marine  Corps  aircraft;  related  weapons  systems;  and  the  support 
establishment  ashore  (DoN,  2012).  Consequently,  any  replenishment  for  stock  that  the  ship 
might  require  to  maintain  operational  readiness  must  be  paid  for  with  0&M,N  dollars.  At 
the  force  level,  these  funds  are  controlled  by  the  type  commander  (TYCOM)  and  allocated 
down  to  the  unit  supply  officer  on  a  quarterly  basis  in  the  form  of  an  operating  target 
(OPTAR).  The  OPTAR  will  pay  for  all  spare-part  requirements  and  any  related  costs,  to 
include  transportation  costs,  ordering  costs,  and  any  holding  costs  associated  with  keeping 
the  part  in  inventory. 

I.  ALLOWANCING  PRODUCT  MAKEUP 

1.  Depot-Level  Repairable  Versus  Non-Depot-Level  Repairable 

The  typical  allowancing  package  is  composed  of  a  variety  of  different  line  items  and 
quantities  dependent  on  the  type  of  equipment  onboard  the  ship,  the  dollar  value,  the  required 
endurance  level,  and  the  average  MTBF.  The  most  basic  classification  among  the  varying 
spare  parts  in  the  allowancing  package  is  the  designation  as  a  depot-level  repairable  (DLR) 
item  versus  a  non-DLR  item,  otherwise  known  as  a  consumable  item. 

DLR  items  are  usually  the  high-dollar-value  components  and  are  deemed  too  costly  to 
dispose  of.  This  classification  requires  that  when  a  DLR  is  unserviceable,  it  must  be  sent 
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back  to  the  pre-determined  repair  facility  for  overhaul  and  subsequent  clearance  as  ready  for 
issue  (RFI).  When  considering  the  dollar  value  of  these  items,  the  U.S.  Navy  aggressively 
tracks  the  movement  and  repair  status  of  all  items  both  on  the  vessels  and  ashore  to  eliminate 
unnecessary  waste.  As  a  rule  of  thumb,  the  allowance  for  a  DLR  in  an  allowancing  package 
is  incapable  of  being  altered  by  the  personnel  onboard  the  ship.  These  allowances  are  to 
remain  as  they  were  when  they  were  implemented.  The  designation  of  DLRs  is  determined 
by  NAVSUP  guidance  (NAVSUP,  1997).  Specific  codes  are  assigned  to  segregate  the  DLRs 
by  their  application  of  use  in  a  particular  system  and  into  manageable  groupings  based  on 
their  level  of  usage.  In  addition,  these  codes  are  used  to  designate  which  inventory  manager 
will  handle  the  day-to-day  administration  of  that  particular  item. 

The  non-DLR  items  are  not  tracked  as  closely  but  still  require  a  compulsory  level  of 
oversight.  Consumable  items  are  kept  onboard  the  ship  to  be  used  as  necessary;  once  the  life 
of  that  asset  has  been  exhausted,  the  item  is  simply  discarded  in  accordance  with  published 
guidelines.  The  logistics  personnel  onboard  the  ships  are  given  authorization  to  manipulate 
the  allowance  of  these  items  as  they  see  fit  to  meet  future  demand  signals.  The  following  is  a 
list  of  the  most  commonly  found  allowance  classifications  for  non-DLR  items:  9C,  9B,  9G, 
9N,  and  9E  (NAVSUP,  1997). 

2.  Allowance  Type  Codes 

When  apportioning  allowances,  the  allowancing  package  will  assign  a  classification 
code  to  the  assets  within  the  package.  This  will  enable  the  logistics  personnel  to  know  by 
which  process  the  asset  was  placed  on  the  ship.  The  classification  codes  are  broken  down 
into  nine  different  categories,  called  allowance  type  (AT)  codes.  The  most  common 
classification  with  regard  to  allowancing  products  is  an  ATI.  This  classification  explains 
that  the  asset  is  mandated  to  be  carried  because  of  the  implementation  of  an  allowancing 
product.  A  breakdown  of  the  other  applicable  AT  codes  and  descriptions  are  captured  in 
Table  1. 
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Table  1.  Allowance  Type  Codes 
(NAVSUP,  1997) 


AT  Code 

Name 

Description 

1 

Cosal  Item 

COSAL  item. 

2 

Aviation  Support  Item 

Load  List  item,  applicable  to  load  carrying  ships  only. 

3 

COSAL/AV  Item 

Load  and  Allowance  List  item,  applicable  to  load  carrying  ships  only. 

4 

Demand  Based  Item  (DBI) 

Non-CO  SAL  item  that  is  stocked  based  solely  on  demand. 

5 

TYCOM  Directed  Item 

Non-COSAL  authorized  add  item  based  on  specific  TYCOM  authority. 

6 

Excess  Item 

Non-COSAL  excess  item  that  does  not  have  sufficient  demand  to  maintain.  The 

item  is  to  be  offloaded  and  deleted. 

7 

Economic  Retention  Item 

Economic  retention.  Excess  item  that,  because  of  low  unit  cost  (normally  under 
$100.00),  is  authorized  for  retention  until  the  next  ILO/ReAVCAL.  The  exact 
amount  may  be  specified  by  each  TYCOM. 

8 

Demand  Recording 

Non-COSAL  item  that  is  established  for  demand  recording  only.  It  will  be  changed 
to  AT  code  4  if  demand  reaches  established  criteria,  otherwise  it  will  be  deleted 
after  24  months  with  no  demand. 

9 

Substitute/Alternate  Item 

Assigned  to  a  substitute  item  which  is  not  stocked  as  a  primary  number. 

3.  Mission  Criticality  Codes 

The  U.S.  Navy’s  Item  Mission  Essentiality  Coding  (IMEC)  system  consists  of  a 
combination  of  military  essentiality  code  (MEC)  and  mission  criticality  code  (MCC).  These 
codes  help  the  allowancing  product  developers  to  apply  weights  to  those  assets  that  are  most 
important  to  the  sustainability  and  readiness  of  the  ship  within  the  allowancing  model.  The 
allowancing  package  contains  either  the  MEC  or  MCC  in  Coordinated  Shipboard  Allowance 
List  (COSAL)  Part  I,  Sections  A  and  B,  depending  on  the  ship’s  computation  method  (DoN, 
2009).  The  possible  MCCs  that  can  be  assigned  to  components  are  listed  in  Table  2. 
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Table  2. 


Mission  Criticality  Codes 
(DoN,  2009) 


CODE  DEFINITION 

1  Failure  of  component/equipment  causes  minor  mission  inpact. 

^  Failure  of  component/equipment  causes  total  loss  or  severe 

degradation  of  a  secondary  mission. 

^  Failure  of  component/equipment  causes  severe  degradation  of  a 

primary  mission  capability. 

Failure  of  component/equipment  causes  total  loss  or  severe 
4  degradation  of  mobility  or  primary  mission  (propulsion  or  life 
support). 

The  loss  of  this  equipment  results  in  a  safety  hazard  to  the  ship  or  its 
crew. 

X  Assigned  to  all  Allowance  Equipage  Lists  (AEL). 


J.  ALLOWANCING  INPUT  AND  OUTPUT 

1.  Inputs 

The  process  for  developing  an  allowance  package  requires  several  pieces  of  data  from 
varying  stakeholders  within  the  process.  The  information  necessary  to  effectively  determine 
the  allowancing  package  ranges  from  configuration  management  data  to  maintenance  history 
data  located  within  the  ship’s  Maintenance,  Material  and  Management  (3M)  database. 

The  configuration  management  data  is  taken  from  the  Configuration  Data  Managers 
Database-Open  Architecture  (CDMD-OA),  which  is  managed  by  NSLC.  CDMD-OA 
tracks  the  status  and  maintenance  of  naval  equipment  and  their  related  logistics  items  (e.g., 
drawings,  manuals)  on  ships  and  naval  activities  around  the  world.  The  status  of  a  given 
piece  of  equipment  on  a  ship  determines  what  and  how  many  spare  parts  will  be  stored  on 
that  ship  for  that  equipment,  making  this  tracking  extremely  important  in  terms  of  cost, 
shipboard  space  and  weight,  and  the  A0  of  the  ship  (CDMD-OA,  2012).  The  U.S.  Navy 
identifies  the  required  spare  parts  onboard  U.S.  Navy  vessels  through  the  use  of  allowance 
parts  lists  (APLs).  APLs  provide  support  and  outfitting  for  parts  that  are  required  for  the 
particular  maintenance  action  performed  onboard  the  ship  (Alvarez,  2010). 

The  3M  database  is  a  central  point  on  the  ship  where  all  data  is  kept  related  to 
maintenance  performed.  In  addition,  the  database  contains  a  comprehensive  list  of  all  assets 
that  are  and  should  be  kept  on  the  ship  to  perform  maintenance.  Along  with  the 
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comprehensive  list  is  the  associated  usage  data  for  the  parts.  If  the  ship  is  a  new 
construction,  the  usage  data  from  similar  vessels  across  the  fleet  will  be  used  in  the 
development  of  the  allowancing  package  for  the  new  vessel. 

In  addition  to  gathering  the  maintenance,  demand,  and  configuration  data  from  the 
3M  system,  data  must  also  be  extracted  from  the  ship’s  current  stock  record  file  (SRF).  The 
SRF  contains  the  financial  records  of  the  ship  to  gauge  the  impacts  of  the  previous 
allowancing  product  against  the  current  funding  levels.  This  information  can  be  very  telling 
as  to  whether  the  previous  product  was  effective  at  saving  costly  procurements  and  the 
inclusion  of  unnecessary  carrying  costs. 

The  final  portion  of  information  that  must  be  incorporated  into  the  allowancing 
package  is  the  Weapons  System  File  (WSF)  Level  C,  which  contains  all  provisioning  and 
technical  decisions. 

2.  Outputs 

With  the  consideration  of  the  aforementioned  input  of  configuration,  financial, 
maintenance,  and  demand  data,  the  stakeholders  are  capable  of  applying  this  information  into 
a  sophisticated  mathematical  allowance  model  that  provides  the  optimized  allowancing 
package.  The  model  can  be  manipulated  to  varying  degrees,  permitting  different  types  of 
allowancing  packages  dependent  on  the  desires  of  the  stakeholders.  In  the  next  section,  we 
explore  several  different  types  of  allowancing  packages. 

Once  the  model  produces  the  output,  the  package  is  socialized  among  the 
stakeholders.  Once  it  is  finalized,  the  allowances  are  generated  by  NAVSUP  WSS  Mech  and 
funded  and  released  to  SURFOR  for  issuance  to  the  ship.  The  final  piece  of  the  puzzle  is  that 
the  ship  will  integrate  the  new  product  into  the  ship’s  database  and  drop  the  order 
requisitions  to  stock  the  shelves  accordingly. 

K.  TYPES  OF  ALLOWANCING  PACKAGES 

1.  Comprehensive  Allowancing 

As  previously  discussed,  the  output  product  from  the  allowancing  process  can  be 
tailored  to  focus  on  a  particular  weapons  system  or  set  to  review  the  entire  ship’s  spare  parts 
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support  package.  The  most  common  comprehensive  allowancing  package  is  the  COSAL. 
The  COSAL  provides  both  technical  and  supply  information,  which  makes  it  an  integrated 
logistics  support  (ILS)  document.  It  is  a  technical  document  to  the  extent  that 
equipment/component/part  nomenclatures,  operating  characteristics,  technical  manuals,  and 
so  forth,  are  described  in  APLs  or  allowance  equipage  lists  (AELs;  NAVSUP,  1997). 

Implementation  of  the  COSAL  is  typically  conducted  during  the  initial  outfitting  of 
the  platform  at  the  beginning  of  its  life  cycle.  Factors  determining  the  composition  of  the 
COSAL  are  the  maintenance  philosophy/capability,  support  concept,  logistics  response  time, 
historical  demand,  and  item/system  population  and  redundancy.  Because  the  COSAL  is  the 
primary  listing  for  all  spare  parts  allocated  for  the  ship,  logisticians  routinely  reference  the 
COSAL  in  its  day-to-day  operations.  Purportedly,  the  COSAL  is  designed  to  provide  the 
warship  with  a  sustained  level  of  material  support  for  90  combat  days  without  replenishment 
and  is  tailored  to  a  particular  ship  class  because  each  ship  class  has  a  different  weapons 
system  configuration  (Axinto  &  Giles,  2005). 

Once  implemented  onboard  the  ship,  the  COSAL  begins  to  receive  periodic  updates 
to  support  any  configuration  or  allowancing  modifications.  Because  COSALs  are  costly  to 
implement  and  very  time  intensive,  the  U.S.  Navy  has  developed  a  system  of  implementing 
the  incremental  changes  to  COSALs  by  way  of  an  automated  shore  interface  (ASI).  These 
updates  are  specifically  tailored  to  a  particular  ship  or  class  to  ensure  that  the  COSAL 
remains  up  to  date  with  configuration  changes  or  additions  and  deletions  to  the  allowancing 
package.  The  periodicity  of  ASI  updates  varies  depending  on  the  timing  of  the  changes 
included  in  the  updates  and  the  funding  levels  available  to  support  those  changes.  The 
process  of  using  ASIs  is  far  less  costly  and  less  labor  intensive  on  the  ship  and  the  shore 
facilities. 

2.  Targeted  Allowancing 

Considering  the  significant  costs  involved  with  developing  and  implementing  a 
COSAL,  the  U.S.  Navy  has  worked  towards  minimizing  those  costs  by  employing  a  targeted 
allowancing  process.  Stakeholders  use  this  focused  approach  to  zero  in  on  the  spare-parts 
requirements  that  would  provide  the  greatest  benefit  towards  increased  readiness  of  the 
shipboard  allowancing  package.  The  process  of  targeted  allowancing  follows  a  similar  path 
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to  the  comprehensive  review,  with  one  exception.  A  filter  is  applied  to  the  mathematical 
model  that  lets  only  certain  aspects  of  the  COSAL  be  reviewed  based  on  criteria  set  forth  by 
the  stakeholders.  Once  the  targeted  allowancing  package  is  produced,  the  implementation 
process  is  identical  and  the  funding  follows  a  similar  stream  for  procurement  of  the  assets. 

L.  CONTINUOUS  INTEGRATED  LOGISTICS  SUPPORT-TARGETED 

ALLOWANCE  TECHNIQUE  (CILS-TAT) 

1.  Overview 

One  of  the  common  targeted  allowancing  techniques  that  the  U.S.  Navy  has  adopted 
is  the  process  termed  the  CILS-TAT.  A  CILS-TAT  is  used  to  focus  the  allowancing  process 
to  minimize  the  overall  funding  requirement,  increase  support  for  poor-performing 
equipment,  and  reduce  allowance  chum  while  providing  the  greatest  increase  to  readiness  for 
the  ship.  In  the  current  fiscally  constrained  environment,  this  approach  is  absolutely  vital  to 
ensuring  that  the  dollars  are  spent  optimally. 

The  U.S.  Navy’s  Maritime  Allowance  Working  Group  (MAWG)  began  discussing 
the  use  of  the  CILS-TAT  in  1999  with  15  prototype  ships.  The  concept  was  to  develop  a 
technique  by  which  ship-optimized  allowances  were  discreetly  applied  to  update  a  ship’s 
SRF,  eliminating  allowances  with  no  usage  and  targeting  systems  with  usage  for  allowance 
updates  of  either  range  additions  or  depth  increases.  During  the  prototype  phase,  the  MAWG 
measured  success  by  looking  at  the  increase  in  supply  effectiveness,  reduction  in  cost,  and 
chum  of  the  allowances.  Through  evaluation  of  these  metrics  specifically,  the  MAWG 
recognized  a  47%  and  50%  reduction  in  the  line-item  churn  and  cost  of  new  allowances, 
respectively,  during  the  prototype  phase  (Bruno,  1999).  The  results  of  the  prototypes  also 
revealed  a  0.9%  increase  in  the  supply  gross  issue  effectiveness,  from  51.8%  to  52.7% 
(Bmno,  1999). 

From  1999  to  2009,  the  U.S.  Navy  adopted  CILS-TAT  as  one  of  the  primary 
methods  for  conducting  targeted  allowancing  and  has  implemented  the  process  on  many 
different  platforms,  to  include  Los  Angeles-class  fast-attack  submarines  (SSNs),  Arleigh 
Burke-class  DDGs,  Avenger-class  mine  countermeasures  ships  (MCMs),  Oliver  Hazard 
Perry-class  frigates  (FFGs),  and  Ticonderoga-class  cruisers  (CGs).  The  selection  process  for 
implementation  of  a  CILS-TAT  on  a  ship  is  conducted  by  various  stakeholders  included  in 
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the  MAWG,  to  include  representatives  from  NSLC,  NAVSUP  WSS  Mech,  NAVSEA,  and 
the  TYCOM. 


2.  Time  Line 

The  time  line  for  implementation  typically  follows  the  deployment  cycles  and  is 
displayed  in  Figure  5.  After  a  ship  has  completed  two  years  of  normal  underway  operations, 
data  is  collected  from  that  period  and  evaluated  by  the  MAWG  to  detennine  whether  the  ship 
is  a  candidate  for  a  CILS-TAT.  As  the  data  is  assessed,  several  key  factors  influence  the 
MAWG’s  decision,  ranging  from  the  availability  of  time  for  implementation  (i.e., 
maintenance  availability  opportunities),  a  recognized  reduction  of  the  supply  gross  issue 
effectiveness,  a  decrease  in  readiness  levels  below  the  fleet  average,  and  funding  availability. 


Figure  5.  CILS-TAT  Implementation  Time  Line 

Once  the  CILS-TAT  is  completed  and  ready  for  implementation,  the  product  is  given 
to  the  TYCOM,  who  is  then  assigned  the  responsibility  of  working  with  the  operational  unit 
to  implement  the  product  into  the  ship’s  onboard  databases.  In  an  effort  of  continuous 
evaluation,  at  the  conclusion  of  another  two-year  period,  the  MAWG  will  once  again  review 
the  health  of  the  ship  and  determine  whether  the  process  was  profitable  and  whether  another 
CILS-TAT  product  should  be  introduced. 

3.  Input 

In  order  to  develop  the  CILS-TAT,  product  stakeholders  draw  from  several  different 
systems  and  add  a  filter  into  the  process  to  focus  on  a  narrowly  defined  set  of  data.  The 
systems  used  in  the  CILS-TAT  are  the  same  as  those  used  to  formulate  a  comprehensive 


vr^vTu-sm  f'K  vcif u 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-22- 


allowancing  product  and  include  the  CDMD-OA,  3M  data  from  the  ship,  CASREP 
information,  and  the  WSF.  Once  this  information  is  compiled,  it  is  run  through  an  elaborate 
mathematical  allowancing  model.  The  final  piece  in  the  application  is  a  filter  that  focuses  the 
allowancing  package  to  look  at  only  a  particular  type  of  3M  data.  Once  the  filter  is  applied, 
the  output  file  is  generated  and  reviewed.  The  CILS-TAT’s  filter  only  (a)  allows  new 
allowances  to  APLs  that  have  had  3M  usage,  (b)  provides  allowance  deletions  or  decreases 
for  those  APLs  with  no  reported  usage,  or  (c)  provides  deletions  for  parts  that  no  longer 
support  installed  configurations.  This  filter  is  what  makes  this  allowancing  product  a  more 
targeted  approach. 

4.  Output 

The  output  provided  by  the  model  is  an  allowancing  package  that  truly  focuses  on 
improving  the  collection  of  parts  necessary  to  support  the  ship  based  on  the  demand  that  has 
been  realized  over  the  previous  two  years.  Those  systems  that  have  needed  a  particular  asset 
onboard,  but  the  part  that  was  not  available  would  now  be  added  to  the  ship’s  inventory, 
thereby  improving  the  supply  gross  issue  effectiveness  and  operational  readiness. 
Specifically,  the  output  file  would  include  (a)  range  additions  to  prevent  missing  a  future 
demand  and  (b)  range  and  depth  decreases.  Those  parts  that  historically  have  not  been 
demanded  or  do  not  currently  conform  to  the  current  configuration  onboard  would  be 
offloaded  through  attrition.  This  more  focused  approach  truly  reduces  the  cost  of 
allowancing,  especially  when  calculating  the  cost  of  allowancing  over  the  entire  U.S.  Navy 
fleet  while  providing  the  most  efficient  use  of  dollars  to  improve  overall  operational 
readiness. 

5.  CILS-TAT  Process  Flow 

The  actual  process  of  a  CILS-TAT  begins  with  the  collection  of  24  months  of 
operational  data  (see  Figure  5).  The  historical  demand  data  is  then  added  to  the  ship’s 
current  COSAL  and  analyzed  for  allowance  additions  and  decreases.  CILS-TATs  are  ship 
specific,  meaning  that  no  two  ships  will  experience  exactly  the  same  types  or  frequency  of 
material  demands.  There  is  a  degree  of  commonality  among  demands,  but  due  to  different 
missions  performed,  environmental  factors  experienced  in  the  area  of  operation,  and  the 
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overall  material  condition  of  the  ship,  these  CILS-TATs  have  to  be  perfonned  on  an 
individual  basis. 

Once  the  24  months  of  data  is  drawn  down  from  the  ship,  the  demands  registered  for 
scheduled  and  unscheduled  maintenance  are  analyzed.  A  CILS-TAT  will  also  add  new 
allowances  when  a  system  configuration  change  is  in  process  or  expected  to  happen  in  the 
near  future.  Allowances  will  also  be  added  to  the  SRF  in  anticipation  of  required 
maintenance  for  each  of  those  new  systems. 

Once  analysis  of  both  the  historical  demand  data  and  forecasted  equipment  upgrades 
are  complete,  the  new  allowances  and  the  remainder  of  the  COSAL  product  is  sent  to  the 
ship  for  loading  into  the  central  database.  The  CILS-TAT  output  is  then  validated  by 
comparing  the  total  list  of  national  item  identification  numbers  (NIINs)  recommended  by  the 
CILS-TAT  against  the  allowances  that  the  ship  already  has  onboard.  Any  shortfalls 
identified  during  this  validation  become  candidates  for  procurement.  To  simplify  this  with 
an  example,  a  CILS-TAT  may  recommend  that  the  ship  carry  5,000  different  repair  parts  at  a 
cost  of  $2.5  million  but,  after  validating  the  CILS-TAT  against  the  ship’s  current  onboard 
allowances,  determines  that  only  300  new  parts  at  a  cost  of  $250,000  may  be  additionally 
required.  A  CILS-TAT  not  only  recommends  new  allowances  but  also  allows  the  ship  to 
validate  its  current  configuration. 

Once  all  shortfalls  are  identified,  requisitions  are  prepared  for  each  new  NUN  to  be 
procured.  The  ship  is  not  required  to  prioritize  the  new  allowance  requisitions  because  all 
additional  allowances  are  considered  valid  by  NSLC  and  will  be  further  evaluated  by  the 
supply  system  during  the  requisition  filling  process.  There  are  three  distinct  phases  of  further 
evaluation  that  each  requisition  goes  through  before  funding  is  obligated.  We  cover  these 
phases  in  the  next  section. 

6.  Requisition  Screening 

Once  requisitions  are  released  into  the  supply  system  as  a  result  of  a  CILS-TAT,  they 
are  reviewed  by  the  NAVSUP  Outfitting  Support  Activity  (OS A)  and  Outfitting  Requisition 
Control  Accounting  System  (ORCAS)  managers.  The  purpose  of  this  screening  is  to  conduct 
quality  checks  and  ensure  that  the  correct  fund  codes  and  advice  codes  were  applied  to  each 
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requisition  by  the  ship.  These  codes  play  an  important  role  in  determining  the  right  type  of 
funding  applied  to  each  of  the  requisitions  and  confinning  that  the  obligations  and 
expenditures  eventually  match  for  each.  Requisitions  flow  freely  through  this  process  and 
are  not  held  up  for  funding  constraints.  The  main  concern  here  is  to  ensure  that  the 
requisition  was  submitted  properly  by  the  ship. 

Once  the  requisitions  complete  the  quality  assurance  process,  they  then  move  to  the 
Pushed  Material  Inventory  Control  System  (PMICS)  offices,  which  are  a  branch  within 
NSLC.  The  new  allowances  recommended  by  the  CILS-TAT  are  compared  against 
allowances  that  are  already  being  procured  via  other  means — for  example,  the  In-Service 
Engineering  Activity  (ISEA).  A  contractor  may  already  have  the  responsibility  to  provide 
initial  outfitting  support  for  a  particular  system  or  group  of  systems  onboard  the  ship.  The 
screen  through  PMICS  is  done  to  look  at  all  possible  avenues  from  which  this  support  could 
come  and  ensure  that  the  government  is  not  paying  for  the  same  initial  allowance  for  the 
same  ship  twice.  Requisitions  can  sometimes  be  held  up  in  this  process  if  the  ship  is  entering 
a  maintenance  availability  or  shipyard  period  and  material  deliveries  to  the  ship  are  being 
routed  to  another  location. 

Once  a  requisition  clears  the  PMICS  and  NAVSUP  OS  A  screens,  requisitions  are 
split  between  high  and  low  value.  (High  value  is  any  requisition  with  an  extended  value  of 
more  than  $300.)  During  periods  when  defense  budgets  were  not  so  tight,  low-value 
requisitions  would  then  immediately  be  funded  and,  if  material  were  available,  they  would  be 
sent  to  the  ships.  High-value  requisitions  are  caught,  and  a  re-evaluation  of  each  allowance 
is  conducted  to  ensure  that  the  allowance  is  still  valid  before  funding  is  applied.  Because  the 
process  up  to  this  point  can  take  up  to  45  days,  it  is  possible  that  a  new  allowance  identified 
by  the  CILS-TAT  may  have  become  obsolete  due  to  an  onboard  system  upgrade  or  a  NIIN 
supersession.  The  NSLC  may  also  cancel  a  requisition  during  this  phase  if  the  re-evaluation 
determines  that  the  part  in  question  no  longer  computes  for  an  allowance  or  that  the  part  has 
already  been  provided  by  the  NSLC  or  the  ISEA. 

If  a  requisition  is  cancelled  in  error,  the  ship  is  required  to  contact  the  NSLC  or  the 
item  manager  directly  to  have  the  requisition  reinstated.  At  the  time  of  this  report,  all 
requisitions,  both  high  and  low  value,  generated  as  a  result  of  a  CILS-TAT  are  initially  given 
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a  status  code  of  “NM”  by  NSLC,  although  only  high-value  requisitions  are  re-evaluated.  The 
NM  status  code  signifies  that  the  requisitions  are  being  delayed  by  NSLC  until  funding 
becomes  available. 

7.  Prioritization  of  Requisitions 

Once  funding  becomes  available  for  these  new  allowances,  it  is  applied  and  material 
can  start  moving  to  the  ship.  NSLC  does  not  determine  which  requisitions  are  funded  or 
which  CILS-TATs  are  funded  first.  There  may  be  CILS-TAT  requisitions  for  more  than  one 
ship  that  are  awaiting  funding;  therefore,  a  determination  has  to  be  made  concerning  which 
CILS-TATs  are  funded  first.  As  previously  stated,  the  determination  of  priorities  at  this 
level  is  done  by  recommendations  from  the  TYCOM.  The  TYCOM  typically  sets  its 
priorities  based  on  which  ship  is  set  to  deploy  next  and  the  relative  importance  of  the  mission 
that  it  is  set  to  carry  out.  CILS-TAT  requisitions  are  funded  at  this  point  on  an  all-or-nothing 
basis,  meaning  that  if  funds  are  available  for  an  entire  CILS-TAT  output  for  a  particular 
ship,  they  are  released.  Otherwise,  requisitions  continue  to  hold  with  an  NM  status  until 
additional  funds  are  available.  No  prioritization  exists  for  the  types  (or  criticality)  of  a 
particular  part  over  another,  and  as  a  consequence,  allowances  that  have  little  impact  on 
overall  system  readiness  (as  determined  by  the  MCC)  may  be  funded  ahead  of  those  that 
could  cripple  a  primary  mission  area. 

M.  NAVY  ENTERPRISE  RESOURCE  PLANNING 

1.  Overview 

In  an  effort  to  improve  the  functionally  and  compatibility  of  such  a  wide 
conglomerate  of  information  systems  across  the  U.S.  Navy  enterprise,  senior  leaders 
endeavored  to  find  an  information  software  package  that  could  help  the  U.S  Navy  streamline 
the  process  of  logistics  information  sharing.  They  were  able  to  work  with  commercial 
vendors  to  develop  a  system  that  mirrors  the  civilian  equivalent  of  ERP.  The  Navy  ERP 
program  uses  a  product  from  SAP  Corporation,  the  largest  provider  of  ERP  solutions  in  the 
world. 

Navy  ERP  is  the  DoN  financial  system  of  record,  meaning  that  it  provides  reliable 
information  for  naval  leadership  to  keep  the  fleet  moving  forward.  Navy  ERP  streamlines  the 
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U.S.  Navy’s  business  operations,  namely  financial  and  supply  chain  management  (U.S.  Navy 
ERP,  2013).  In  2010,  Navy  ERP  Release  1.1  (Single  Supply  Solution)  went  live  on  March 
17  at  NAVSUP,  enhancing  the  ability  for  U.S.  Navy  supply  chain  managers  to  effectively 
and  efficiently  provide  Sailors  and  ships  with  the  items  that  they  need  every  day  (NAVSUP, 
2010). 


2.  Impact  on  the  CILS-TAT 

Due  to  the  roll-out  of  Navy  ERP  in  2010,  all  inventory  management  programs  were 
placed  in  a  phasing  plan  to  gradually  implement  the  system  and  mitigate  any  adverse  effects 
that  might  occur  by  rushing  through  the  implementation  process.  The  highest  priority  was 
given  to  the  requirements  that  the  fleet  needed  on  a  day-to-day  basis,  thereby  relegating  all 
re-allowancing  processes  to  a  lower  priority  until  the  system  had  been  proven.  This  measure 
was  taken  to  ensure  that  no  high-priority  requirements  were  missed  during  the  migration 
period.  The  measure  led  to  the  suspension  of  all  CILS-TATs  in  2009  until  further  notice. 

The  Navy  ERP  system  contains  several  improved  capabilities  that  will  significantly 
impact  the  effect  of  the  CILS-TAT  in  the  future.  The  integrated  processes  within  Navy  ERP 
use  a  single  set  of  data,  automatically  disseminate  information  from  one  entry  to  all  parts  of 
the  process  where  it  is  required,  and  make  the  entire  end-to-end  information  stream  visible  to 
managers  with  responsibility  over  the  processes  (U.S.  Navy  ERP,  2013).  There  are  upgraded 
supply  chain  and  financial  management  capabilities  that  provide  for  better  asset  visibility 
across  the  enterprise  as  well  as  better  reporting  of  the  financial  impacts  of  CILS-TAT 
implementation.  The  improved  supply  chain  suite  also  aids  in  acquiring  the  parts  to  fill  the 
newly  added  allowances. 

As  of  2013,  Navy  ERP  has  proven  to  be  a  significant  success  across  the  U.S.  Navy, 
and  senior  leadership  within  the  logistics  community  is  once  again  ready  to  discuss  the  return 
of  the  CILS-TAT.  If  CILS-TAT  is  proven  to  be  a  wise  investment  and  re-implemented,  it 
should  benefit  greatly  from  the  increased  functionality  recognized  through  the  use  of  Navy 
ERP. 
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II.  METHODOLGY 


A.  INTRODUCTION 

Included  in  the  following  chapter  is  a  brief  overview  of  the  methodologies  or  models 
used  in  conducting  the  BCA  regarding  the  effectiveness  of  CILS— TAT.  The  reasoning 
behind  examining  the  following  models  was  threefold.  First,  we  utilized  the  cost-benefit 
analysis  (CBA)  to  attach  a  dollar  amount  necessary  to  achieve  a  certain  level  of  readiness. 
Further,  a  CBA  allowed  us  to  determine  the  cost  and  benefits  to  be  weighed  against  a 
performance  metric  defined  by  the  stakeholder.  Next,  we  developed  a  regression  model  to 
evaluate  the  effectiveness  of  CILS— TAT  from  the  before  and  after  periods.  The  regression 
model  was  further  refined  through  the  use  of  cluster  analysis,  which  enabled  the  grouping  of 
data  in  an  attempt  to  find  commonality  experienced  by  varying  sets  of  ships.  Finally,  we 
include  in  this  chapter  an  overview  of  the  knapsack  model.  This  information  is  contained 
within  this  chapter  for  use  in  making  improvements  to  the  current  CILS— TAT  framework. 
The  aforementioned  methods  provide  a  comprehensive  set  of  tools  for  developing  a  thorough 
BCA. 

B.  BUSINESS  CASE  ANALYSIS 

1.  Background 

In  business  and  throughout  the  DoD,  the  use  of  BCA  has  gained  popularity  through 
the  leadership  of  the  Office  of  the  Secretary  of  Defense  (OSD)  to  support  strategy  decisions 
regarding  program  implementation.  A  BCA  provides  a  best-value  analysis  that  considers  not 
only  cost  but  also  other  quantifiable  and  non-quantifiable  factors  supporting  an  investment 
decision  (DAU,  2013).  Depending  on  the  type  of  BCA,  it  may  be  used  throughout  the  life 
cycle  of  the  project.  Specifically,  the  BCA  should  be  used  in  further  decisions  to  sustain  or 
enhance  the  solution  and  to  refine  estimation  of  benefits  and  costs  for  future  projects  in  the 
organization  (DAU,  2013).  The  BCA  can  be  thought  of  as  an  expanded  CBA  with  the  intent 
of  detennining  a  best-value  solution.  The  BCA  process  goes  beyond  cost/benefit  or 
traditional  economic  analyses  by  documenting  how  each  alternative  fulfills  the  strategic 
objectives  of  the  program  and  the  resulting  impact  on  stakeholders  (DAU,  2013). 
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Within  the  BCA,  a  CBA  is  used  as  a  method  to  aid  decision-makers  in  quantifying 
differences  between  projects.  It  provides  a  comparative  assessment  of  all  benefits  anticipated 
and  the  costs  incurred  in  various  iterations.  CBAs  thereby  allow  decision-makers  to  pick  the 
optimal  solution  for  the  allocation  of  scarce  resources. 

When  conducting  a  CBA,  three  frames  of  reference  can  be  used  to  approach  the 
analysis:  ex  ante,  ex  post,  and  in  medias  res  (Boardman,  1996,  p.  3).  These  different 
reference  frames  define  the  time  at  which  the  analysis  is  completed.  In  ex  ante  analysis,  a 
CBA  is  conducted  prior  to  the  start  of  a  project.  The  advantage  of  ex  ante  analysis  is  that  it 
facilitates  the  decision-maker  in  ensuring  better  decisions  about  the  allocation  of  resources; 
however,  it  does  not  provide  the  most  complete  understanding  of  the  actual  benefits  or  cost 
assumed — there  is  a  high  degree  of  uncertainty.  An  in  medias  res  analysis  is  completed  as 
the  project  progresses.  Although  this  method  provides  a  reduction  of  uncertainty,  it  reduces 
the  ability  of  the  decision-maker  to  have  full  control  of  the  proper  allocation  of  scarce 
resources.  If  a  project  has  been  completed,  an  ex  post  CBA  can  be  conducted.  This  method 
does  not  provide  a  decision-maker  with  the  ability  to  allocate  resources  because  they  have 
already  been  expensed,  but  it  is  the  most  accurate  method  to  understand  the  actual  impacts  of 
benefits  and  costs.  For  most  companies,  ex  post  analysis  is  not  an  option  because  of  the  large 
monetary  investment  in  conducting  a  CBA.  In  general,  most  companies  or  government 
agencies  rely  on  ex  ante  analysis  with  an  understanding  that  there  is  an  added  degree  of  risk 
due  to  inherent  uncertainties.  Looking  further  into  the  CBA  process,  the  framework  for  the 
analysis  consists  of  nine  steps  (Boardman,  1996,  pp.  6-24).  The  following  is  a  breakdown  of 
each  step: 

2.  The  Steps  of  a  Cost-Benefit  Analysis 

The  following  steps  describe  a  typical  CBA: 

1 .  Decide  whose  benefits  and  costs  count.  In  this  step,  the  analysts  must  choose 
the  scope  of  the  analysis  and  determine  the  target  group.  Specifically,  the  analysts  must 
decide  whether  they  are  looking  from  the  perspective  of  a  guardian.  This  decision  becomes 
integral  in  later  steps  because  it  will  define  how  costs  or  benefits  are  viewed — a  benefit  to  a 
guardian  may  be  a  cost  to  a  spender.  In  general,  guardians  ignore  nonfinancial  social 
benefits.  For  social  benefits  in  the  federal  government,  it  is  assumed  that  the  analysis  is 
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being  completed  from  the  spender’s  (society’s)  perspective  vice  the  guardian’s  (federal 
government’s)  perspective  (OMB,  1992). 

2.  Select  the  portfolio  of  alternative  projects.  This  step  allows  the  analysts  to 
bound  the  project  being  analyzed.  For  simplicity,  only  one  project  is  analyzed  at  a  time — in 
theory,  there  are  infinite  numbers  of  alternatives.  This  step  mitigates  uncertainty  associated 
with  complicated  relationships  among  parameters  being  manipulated.  As  the  number  of 
possible  values  increases  among  alternative  projects,  the  overall  alternatives  increase 
exponentially.  As  Boardman  (1996)  observed,  “If  there  were  n  dimensions,  each  with  k 
possible  values,  there  would  be  kn  alternatives  ...  there  would  be  27  mutually  exclusive 
alternatives.  Neither  decision  makers  nor  analysts  can  cognitively  handle  comparison  among 
such  a  large  number  of  alternatives”  (p.  13).  Often,  external  constraints  further  restrict  the 
project  from  reaching  the  optimal  output  levels.  The  analysts  must  be  cognizant  of  these 
factors  and  what  limitations  are  being  imposed. 

3.  Catalog  potential  (physical)  impacts,  and  select  measurement  indicators. 
Within  this  step,  the  analysts  define  variables,  both  tangible  and  intangible,  that  are  being 
considered  and  quantify  their  impact  to  the  overall  project.  Chosen  variables  must  ensure 
that  there  is  a  cause-and-effect  relationship  between  a  tangible  outcome  and  a  society.  When 
defining  variables,  analysts  must  explicitly  state  all  assumptions,  especially  if  assumptions 
are  made  about  future  benefits  or  costs  (OMB,  1992).  Once  the  list  is  collected,  the  variables 
are  aggregated  as  either  a  benefit  or  a  cost. 

4.  Predict  quantitative  impacts  over  the  life  of  the  project.  This  step  takes  each 
variable  impact  and  projects  its  value  and  changes  over  the  life  of  the  project.  In  some 
instances,  the  projections  are  taken  past  the  life  of  a  project  if  the  project  is  anticipated  to 
have  a  continued  impact  on  alternate  projects  following  termination.  This  section  looks  to 
correlate  the  impact  of  a  variable  to  a  tangible  value — for  example,  lives  saved  per  year  or 
part  reduction  per  year.  If  existing  projects  are  available,  historical  data  can  be  used  as  a 
baseline  to  extrapolate  the  impact  of  possible  changes. 

5.  Monetize  (attach  a  dollar  value  to)  all  impacts.  The  goal  of  this  step  is  to 
associate  a  monetary  amount  to  all  impacts  in  terms  of  the  dollar  amount  saved  or  lost  for 
each  variable.  This  goal  ensures  a  common  set  of  units  for  comparing  one  variable  to 


v-JfNPSTte-j 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-31  - 


another.  The  monetary  amount  associated  with  an  impact  is  often  valued  based  on  a 
willingness  to  pay  (OMB,  1992).  This  relationship  can  be  extrapolated,  for  a  well-defined 
product,  from  market  prices.  Variables  such  as  parts  reduction  or  labor  reduction  will  be 
monetized  based  on  the  market  price  for  labor  or  the  price  of  each  part  reduced. 

6.  Discount  for  time  to  find  present  values.  If  a  project  is  expected  to  have 
benefits  or  costs  realized  over  the  course  of  the  project’s  life  (years),  these  future  values  must 
be  aggregated  in  their  present  value  (PV)  for  comparison.  The  PV  of  a  future  cost  or  benefit 
can  be  calculated  using  the  PV  equation  (see  Figure  6). 


Future  Value  of  Benefit  or  Cost 

Present  Value  = - 7 - — r - 

(1  +  dfi 

Figure  6.  Present  Value  Equation 
(Brealey,  Myers,  &  Allen,  2011,  p.  104) 

In  Figure  6,  d  represents  the  opportunity  cost  of  capital,  and  t  represents  the  number 
of  years  in  the  future  that  the  benefit  or  cost  is  realized  (Brealey  et  ah,  2011,  p.  104).  The 
opportunity  cost  of  capital  represents  the  expected  return  not  realized  because  of  a  project 
investment  compared  to  financial  securities  (Brealey  et  ah,  2011,  pp.  G-l  1). 


7.  Sum:  Add  up  the  benefits  and  costs.  Using  the  PV  of  each  variable,  the 

analysts  sum  up  all  benefits  and  costs  to  calculate  the  net  present  value  (NPV)  of  each 
alternative  (see  Figure  7). 


Net  Present  Value  = 


^  PV  of  Benefits  — 


^  PV  of  Costs 


Figure  7.  Net  Present  Value  Equation 
(Brealey  et  al.,  2011,  p.  104) 

8.  Perform  sensitivity  analysis.  A  sensitivity  analysis  allows  the  analysts  to 
capture  the  impact  of  uncertainty  for  each  variable  within  a  project.  This  analysis  will 
identify  to  a  decision-maker  what  variables  can  absorb  higher  degrees  of  uncertainty  with 
marginal  impacts  to  the  outcome.  It  is  very  rare  that  the  impact  of  a  variable  or  its  valuation 
per  unit  impact  is  known  completely.  A  sensitivity  analysis  can  be  completed  by  either 
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manually  adjusting  variables  to  see  their  impact  to  the  output  or  by  conducting  a  model 
analysis  (Ragsdale,  2008,  p.  136). 


9.  Recommend  the  alternative  with  the  largest  NPV.  When  choosing  between 


alternatives,  the  decision-maker  should  choose  the  project  or  alternative  with  the  highest 
NPV.  If  the  decision  is  between  all  negative  NPVs,  the  project  or  alternative  with  the  lowest 
negative  value  should  be  chosen.  After  the  analysis,  there  may  be  an  instance  where  doing 
the  status  quo  will  result  in  a  higher  NPV — sometimes  doing  nothing  different  is  a  better 
option. 

C.  KNAPSACK  PROBLEM 

1.  Introduction 

Integer  and  combinatorial  optimization  deals  with  the  problems  of  maximizing  or 
minimizing  a  function  of  many  variables  subject  to  (a)  inequality  and  equality  constraints  and 
(b)  integrality  restrictions  on  some  or  all  of  the  variables.  Because  of  the  robustness  of  the 
general  model,  a  remarkably  rich  variety  of  problems  can  be  represented  by  discrete 
optimization  models  (Nemhauser  &  Wolsey,  1988). 

The  knapsack  problem  (KP)  is  founded  in  the  scope  of  integer  and  combinatorial 
optimization.  Suppose  that  a  hitchhiker  has  to  fill  up  his  knapsack  by  selecting  from  among 
various  possible  objects  those  which  will  give  him  maximum  comfort.  This  very 
rudimentary  question  fonnulates  the  basis  of  the  KP.  According  to  Martello  and  Toth 
(1990),  the  KP  can  be  mathematically  formulated  by  numbering  the  objects  from  1  to  n  and 
introducing  a  vector  of  binary  variables  Xj  (j  —  1, ... ,  n )  with  the  following  assignments: 


if  object  j  is  selected ; 

otherwise. 


(2) 


Then,  if  pj  is  a  measure  of  the  comfort  given  by  object  j,  Wj,  its  size,  and  c  (the  size  of  the 
knapsack),  our  problem  will  be  to  select,  from  among  all  binary  vectors  x  satisfying  the 
constraint 


YIj=\WjXj  <  C, 


(3) 


the  one  which  maximizes  the  objective  function 
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Tj=  iPjXj. 


(4) 


The  KP  has  attracted  and  been  intensively  studied  by  both  theorists  and  practitioners. 
Nemhauser  and  Wolsey  (1988)  espoused  that  an  important  and  widespread  area  of 
application  concerns  the  management  and  efficient  use  of  scarce  resources  to  increase 
productivity.  These  applications  include  operational  problems  such  as  the  distribution  of 
goods,  production  scheduling,  and  machine  sequencing.  They  also  include  planning 
problems  such  as  capital  budgeting,  facility  location,  and  portfolio  analysis. 

For  the  purposes  of  this  research,  we  address  a  capital  budgeting  model  with  the 
problem  of  selecting  among  various  allowancing  possibilities  so  as  to  maximize  the  total 
operational  readiness  without  exceeding  the  available  funds.  According  to  Christo fides, 
Mingozzi,  Sandi,  and  Toth  (1979),  this  model  can  be  directly  expressed  as  a  zero-one  (0-1) 
KP  (each  allowance  possibility  is  either  accepted  or  rejected). 

2.  Zero-One  Knapsack  Problem 

The  0- 1  KP  is  the  most  important  KP  and  one  of  the  most  intensively  studied  discrete 
programming  problems.  According  to  Martello  and  Toth  (1990),  the  0-1,  or  binary,  KP  is 
given  a  set  of  n  items  and  the  knapsack,  with 


Pj  —  profit  of  item  j, 


Wj  -  weight  of  item  j 


c  —  capacity  of  the  knapsack. 


Select  a  subset  of  the  items  so  as  to 


maximize 


=  Yg=i  PjXj 


(5) 


subject  to  YJj=\  wjxj  <  c 


(6) 


Xj  —  0  or  1  j  G  N  —  (1, 


where 


1  if  item  j  is  selected; 
0  otherwise. 


(V) 
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The  reason  for  such  an  interest  basically  derives  from  three  facts:  (a)  the  0-1  KP  can  be 
viewed  as  the  simplest  integer  linear  programming  problem,  (b)  it  appears  as  a  subproblem  in 
many  more  complex  problems,  and  (c)  it  may  represent  a  great  many  practical  situations 
(Martello  &  Toth,  1990). 


3.  Bounded  and  Unbounded  Knapsack  Problem 

One  common  variant  of  the  0-1  KP  model  is  that  each  item  can  be  chosen  multiple 
times.  The  bounded  knapsack  problem  (BKP)  specifies  that  for  each  item  j,  an  upper  bound 
bj  (which  may  be  a  positive  integer,  or  infinity)  is  on  the  number  of  times  that  item  j  can  be 
selected  (Martello  &  Toth,  1990).  The  BKP  equation,  as  defined  by  Martello  and  Toth 
(1990),  asserts  to 


maximize 

Z  =  Eyn=1  PjXj 

(8) 

subject  to 

Z"=i  WjXj  <  c, 

(9) 

0  <  Xj  <  bj  and  integer,)  G  N  =  (1,  ...,n}. 

The  BKP  is  a  generalization  of  the  0-1  KP,  in  which  bj  —  1  for  all  j  G  N.  We  assume, 
without  loss  of  generality,  that 


Pj,Wj,bj  and  c  are  positive  integers, 


YJj=i  bj  wj  >  c, 

(10) 

bjWj  <  c  for  j  G  N. 

(11) 

The  unbounded  knapsack  problem  (UKP),  sometimes  referred  to  as  the  integer  KP, 
does  not  put  any  upper  bounds  on  the  number  of  times  that  an  item  may  be  selected 
(Christofides  et  ah,  1979).  This  type  of  scenario  might  be  applicable  to  the  allowancing 
product  in  that  the  allowancing  model  detennines  that  multiple  assets  of  the  same  line  item 
are  necessary  to  support  the  demand  signal  of  the  ship.  According  to  Martello  and  Toth 
(1990),  the  model  seeks  to 


maximize 

N 

II 

M 

V 

(12) 

subject  to 

Z"=i  WjXj  <  c, 

(13) 
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Xj  >  0  and  integer,  j  E  n  —  (1,  ...,n}. 


Through  the  use  of  combinatorial  optimization,  any  possible  revision  of  the  current 
CILS-TAT  model  would  be  streamlined  to  ensure  the  greatest  yield  for  the  financial 
investment.  The  application  of  the  KP  model  to  this  project  is  vitally  necessary  to  maximize 
the  effectiveness  of  the  allowance  products  developed  for  U.S.  Navy  ships. 

D.  REGRESSION  ANALYSIS 

1.  Overview 

Taken  from  a  very  broad  vantage  point,  regression  analysis  can  be  understood  as  a 
statistical  tool  for  the  estimation  of  relationships  between  variables.  It  includes  many 
techniques  for  modeling  and  analyzing  several  variables  when  the  focus  is  on  the  specific 
relationship  between  a  dependent  variable  and  one  or  more  independent  variables.  More 
specifically,  regression  analysis  helps  explain  how  the  typical  value  of  the  dependent  variable 
changes  when  any  one  of  the  independent  variables  is  varied  while  the  other  independent 
variables  are  held  fixed  (Lind  &  Mason,  1993).  Regression  analysis  can  have  varying  uses, 
ranging  from  prediction  (to  include  the  forecasting  of  time-series  data),  inference,  hypothesis 
testing,  and  modeling  of  causal  relationships  (Anderson,  Sweeney,  &  Williams,  2000). 

Linear  regression  was  the  first  type  of  regression  analysis  to  be  studied  rigorously  and 
to  be  used  extensively  in  practical  applications.  According  to  Anderson  et  al.  (2000),  the 
equation  includes  the  effects  or  regression  coefficients  ((3),  a  dependent  variable  (y), 
independent  variables  (x),  and  the  error  term  or  noise  (s).  A  regression  model  relates  the 
dependent  variable  y  to  a  function  of  x  and  P,  written  y  —  f(x,  /?).  The  regression  analysis 
model  can  have  a  single  independent  variable  or  multiple  independent  variables,  as  seen  in 
Figure  8. 


y=p  o  +  ftx  +  £ 


Figure  8.  Linear  Regression  Analysis  Equation 
(Lind  &  Mason,  1993) 

With  the  development  of  a  regression  analysis,  the  researcher  must  develop  a  set  of 
assumptions  regarding  the  probability  distribution  of  the  errors  that  must  hold  in  order  for  the 
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model  to  be  effective.  Statistical  tests  are  then  made  on  the  basis  of  these  assumptions  (Lind 
&  Mason,  1993). 

It  is  worth  mentioning  here  a  couple  of  key  assumptions  that  must  be  considered 
when  evaluating  the  output  from  the  model.  The  first  assumption  that  must  be  addressed  is 
the  presence  of  multicollinearity,  which  is  defined  as  the  problem  where  changes  in  two 
variables  are  nevertheless  highly  correlated — to  the  point  that  it  is  difficult  to  separate  their 
effects  on  the  dependent  variable  (Sykes,  1993).  The  other  assumption  that  must  be 
addressed  is  the  presence  of  normality.  The  evaluation  of  normality  is  done  by  calculating 
the  random  error  in  the  relationship  between  the  independent  variables  and  the  dependent 
variable  in  a  regression  model;  the  random  error  should  be  normally  distributed. 

Once  the  regression  model  is  developed,  the  researcher  must  assess  the  statistical 
significance  of  the  estimated  relationship  to  detennine  the  strength  of  the  relationship 
predicted  by  the  data  against  the  true  relationship.  Along  with  determining  the  statistical 
significance,  the  investigator  must  determine  the  goodness  of  fit  for  the  model  by  evaluating 
the  R~  value.  The  R  value  is  a  number  between  0  and  1  that  describes  how  well  a  regression 
fits  a  data  set.  Once  the  model  is  developed,  assumptions  have  been  validated,  and  the 
statistical  significance  and  goodness  of  fit  has  been  proven,  the  model  is  ready  to  be  used  for 
various  applications  (Anderson  et  ah,  2000). 

2.  Linear  Regression 

When  developing  a  simple  linear  regression,  the  researcher  must  determine  which 
variables  of  interest  must  be  taken  from  the  data  set  to  satisfy  the  question  posed  by  the 
hypotheses.  This  data  will  then  be  graphically  depicted  using  a  scatter  plot  diagram.  The 
resulting  display  will  give  the  researcher  the  ability  to  easily  determine  the  general 
correlation  between  the  two  variables  when  the  other  covariates  are  held  fixed. 

The  regression  is  further  defined  through  the  use  of  the  least  squares  method  to  fit  a 
line  to  the  distribution.  We  call  the  estimate  of  the  line’s  intercept  bo  and  that  of  the  line’s 
slope  bi.  The  estimated  or  predicted  value  of  y  is  denoted  as  y.  Within  the  scatter  plot,  the 
hypothesized  relationship  thus  implies  that  somewhere  on  the  diagram  may  be  found  a  line 
with  the  equation  y  —  b0  +  fyx.  In  the  least  squares  method,  we  minimize  the  sum  of 
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squared  differences  between  y  and  y.  Then  we  define  a  residual  for  observation  i  to  be 
ei  —  yt  —  ji  and  minimize  the  sum  of  squared  errors,  £f=1  et  (Anderson  et  ah,  2000). 

The  same  fundamental  equation  can  also  be  applied  when  multiple  independent 
variables  are  introduced.  The  technique  is  called  multiple  regression,  and  it  allows  additional 
factors  to  enter  the  analysis  separately  so  that  the  effect  of  each  can  be  estimated.  The 
technique  is  valuable  for  quantifying  the  impact  of  various  simultaneous  influences  upon  a 
single  independent  variable  (Sykes,  1993).  Furthermore,  because  of  omitted  variables  bias 
with  simple  regression,  multiple  regression  is  often  essential,  even  when  the  researcher  is 
only  interested  in  the  effects  of  one  of  the  independent  variables  (Sykes,  1993).  The  general 
form  of  the  model  is  y  —  /?0  +  /31x1  +  /?2x2  +  — I-  Ppxp  +  e,  and  the  estimated  relationship 
is  y  —  b0  +  b1x1  +  b2x2  +  — I-  bpxp.  We  use  the  least  squares  method  to  find  the  values  of 
bo,  bi...  bp  that  minimizes  the  sum  of  the  squared  differences  between  yt  —  yt  (Lind  & 
Mason,  1993). 


3.  Assumptions 

As  with  any  model,  there  are  fundamental  assumptions  that  must  be  understood 
before  the  researcher  can  proceed  with  the  interpretation  of  the  results.  There  are  six 
important  assumptions  centered  on  regression  analysis. 

1 .  Relationship  possesses  linearity. 

2.  Error  terms  (e)  are  normally  distributed. 

3.  At  every  x  value,  the  error  terms  have  constant  variance  (homoscedasticity). 

4.  The  error  terms  are  independent  of  each  other. 

5.  There  is  a  lack  of  multicollinearity. 

For  the  regression  to  possess  linearity,  the  mean  of  the  response  variable  is  a  linear 
combination  of  the  parameters  (regression  coefficients)  and  the  predictor  variables.  Because 
the  predictor  variables  are  treated  as  fixed  values,  linearity  is  really  only  a  restriction  on  the 
parameters  (Noether,  1971).  In  order  to  check  for  model  linearity  and  constant  variance,  a 
scatter  plot  would  be  used.  This  is  done  by  plotting  the  residuals  against  the  predicted 
values.  The  researcher  would  hope  to  see  a  plot  that  reveals  no  patterns;  in  other  words,  he 
or  she  wants  the  plot  to  have  a  lot  of  randomly  distributed  points.  If  the  error  terms  have 
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different  variances,  the  researcher  would  see  the  spread  in  the  residuals  changing  as  a 
function  of  the  predicted  value  (Anderson  et  ah,  2000). 

To  test  the  assumption  that  error  terms  are  normally  distributed,  the  researcher  must 
evaluate  the  residuals.  This  process  is  conducted  by  constructing  a  histogram  of  the 
residuals.  If  the  distribution  looks  bell-shaped,  the  researcher  can  feel  comfortable  that  the 
error  terms  are  close  to  normally  distributed  (Anderson  et  ah,  2000). 

The  final  assumption  in  linear  regression  is  that  the  independent  variables  are  truly 
independent  of  each  other.  The  violation  of  this  assumption  is  referred  to  as 
multicollinearity.  If  the  researcher  is  interested  only  in  prediction,  then  multicollinearity  may 
not  represent  a  large  problem.  If  the  researcher  is  trying  to  explain  the  relationships  between 
dependent  and  independent  variables,  it  does  cause  problems  (Lind  &  Mason,  1993). 

The  main  problem  is  that  the  standard  error  of  the  regression  coefficients  is  highly 
inflated;  hence,  the  estimated  regression  coefficients  have  large  sampling  variability. 
Estimated  regression  coefficients  tend  to  vary  widely  from  one  sample  to  the  next  when  the 
independent  variables  are  highly  correlated.  Another  problem  is  the  interpretation  of  the 
estimated  coefficients.  When  the  explanatory  variables  are  correlated,  the  researcher  cannot 
change  one  variable  without  the  correlated  variable(s)  changing  at  the  same  time. 

There  are  two  ways  of  identifying  multicollinearity  within  the  model,  and  the 
approach  varies  depending  on  preference.  The  researcher  can  either  look  for  the  effects  of 
the  correlation  or  focus  on  the  causes.  One  example  of  looking  for  effects  might  be 
identifying  large  changes  in  the  estimated  regression  coefficients  when  a  variable  is  added  or 
deleted.  An  example  of  looking  for  causes  would  be  to  recognize  large  correlation 
coefficients  between  independent  variables  in  the  correlation  matrix  (Neter,  Wasserman,  & 
Kutner,  1990). 

4.  Statistical  Inference  and  Goodness  of  Fit 

The  process  of  determining  the  statistical  inference  of  the  model  is  done  by  looking  at 
the  goodness  of  fit  of  the  model  and  whether  or  not  the  overall  relationship  is  significant.  In 
other  words,  is  the  dependent  variable  related  to  any  of  the  independent  variables?  Variables 
are  evaluated  by  testing  a  hypothesis  of  Ho  and  Ha.  The  hypothesis  is  tested  by  comparing 
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the  amount  of  variation  explained  by  the  independent  variables  to  the  amount  of  variation  left 
unexplained.  The  unexplained  variance  is  the  residual  mean  square.  The  explained  portion 
is  referred  to  as  the  regression  mean  square,  and  the  F  statistic  is  the  ratio  of  explained  to 
unexplained  variance  (Anderson  et  ah,  2000). 

All  parameters  were  analyzed  using  the  Welch  Two-Sample  /-test.  This  test  method 
assumes  unequal  variance  between  data  sets  and  that  the  two  data  sets  are  not  paired.  Under 
the  assumption  of  unequal  variances,  the  denominator  of  the  /-test  is  not  a  function  of  the 
pooled  variance,  as  would  be  the  case  in  the  Student’s  t-test  (Keller,  2009).  Equations  14  and 
15  give  the  equation  for  the  Welch  t-statistic. 


X,-X2 


(14) 


v  = 


+ 


tfiW-1)  n2\n2-i) 


(15) 


This  test  was  completed  using  the  /-test  function  in  the  stats  package  of  R  Studio 
software.  Each  data  set  was  evaluated  using  three  different  hypotheses  tests: 


Two  Sided  < 


H0  •  M Before  A After  ^ 

H1  :  A Before  ~  A After  *  0 


(16) 


Less  < 


H0  -A Before  M After  ^ 
-A Before  ~  M After  <  ^ 


(17) 


Greater  < 


H0  '^Before  M After  ~  ^ 
Hi  -M Before  ~  A After  >  ^ 


(18) 


Using  these  three  tests  ensures  that  any  movement,  positive  or  negative,  in  the  data 
set  means  is  fully  captured  in  the  analysis.  In  each  case,  a  95%  confidence  interval  (Type  I 
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error  <  0.05)  was  used  as  a  baseline  metric  to  establish  statistically  significant  changes  in  the 
data  sets. 

E.  CLUSTER  ANALYSIS 

1.  Overview 

Cluster  analysis  is  a  subset  of  data  analysis  tools  called  data  mining.  This  analysis 
seeks,  by  grouping  information,  to  find  the  interrelationships  across  various  parameters  or 
variables  to  understand  the  structure  of  data  sets.  Cluster  analysis  places  data  of  similar 
values  together  to  create  a  series  of  A  homogeneous  groupings  (Ye,  2003).  For  the  purposes 
of  this  project,  in  an  effort  to  understand  why  certain  ships  experienced  a  statistically 
significant  change  in  allowance  effectiveness,  cluster  analysis  is  used  to  find  the  relationship 
among  various  source  codes  and  allowance  effectiveness.  For  example,  for  the  ships  that 
achieved  a  statistically  significant  change  in  allowance  effectiveness,  was  it  the  respective 
change  in  source  code  A  or  source  code  G  that  caused  the  overall  change  in  allowance 
effectiveness?  What  are  the  differentiating  factors  among  the  ships?  Cluster  analysis  will 
show  the  thread  among  parameters  that  causes  a  ship  to  see  statistically  significant  change  in 
allowance  effectiveness. 

2.  K-means  Algorithm 

K-means  falls  under  the  partitional  method  of  cluster  analysis.  In  this  analysis,  data  is 
split  into  user-defined  K  clusters.  The  correct  number  of  clusters  is  a  subjective  value, 
depending  on  constraints  within  the  application  or  oftentimes  found  by  iterating  through 
various  possibilities  to  find  the  best  match  (Ye,  2003).  The  overall  goal  is  to  find  a  K  value 
that  partitions  the  data  into  rational  groupings  with  minimum  overlap  among  clusters  and  no 
empty  clusters. 

The  k-means  algorithm  uses  a  five-step  process,  seen  in  Figure  9,  to  identify  the 
location  and  size  of  the  clusters.  In  the  first  step,  K  number  of  cluster  seed  points  are 
identified  from  the  data  set  and  represent  an  initial  estimate  of  the  location  of  the  center  of 
the  clusters  (initial  centroids). 
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Basic  K-means  algorithm 


1 :  Select  K  points  as  initial  centroids. 

2:  repeat 

3: From  K  cluster  by  assigning  each  point  to  its  closest  centroid. 

4: Recompute  the  centroid  of  each  cluster. 

5:  until  Centroids  do  not  change 

Figure  9.  Cluster  Analysis — The  Basic  K-means  Algorithm 

(Ye,  2003) 

Once  the  location  of  the  clusters  has  been  identified,  each  of  the  data  points  is  then 

associated  with  a  respective  cluster  based  on  its  proximity  to  the  cluster  centers. 

Fundamentally,  the  proximity  calculation  is  an  optimization  problem  whose  objective 
function  is  to  minimize  the  sum  of  the  squared  error  (SSE),  Equation  19,  based  on  the 
Euclidian  distance  of  each  point  to  the  cluster  center  (Ye,  2003). 

K 

minimize:  SSE  =  (19) 

z=l  xeC. 

Turning  to  geometry,  the  Euclidian  distance  represents  nothing  more  than  the  shortest 
distance  between  two  points.  In  n-dimensional  space,  this  distance  is  equal  to  Equation  20, 
where  c;  and  x;  represent  the  center  point  of  the  cluster  and  x  represents  a  point  in  the  data  set: 

dist  (ci’xi)  =  J'ECcJ.-xJf  (20) 

Once  the  respective  values  have  been  assigned  to  one  of  the  clusters,  a  new  centroid  is 
calculated  based  on  the  mean  of  the  values  within  the  cluster.  This  process  is  repeated  until 
there  is  no  change  in  the  location  of  the  centroids. 

The  results  of  the  K-means  analysis  are  a  series  K  of  centroid  points  for  each  cluster 
along  with  standard  deviations  of  the  ellipses  representing  a  specified  confidence  interval. 
All  cluster  analyses  in  this  project  were  completed  using  the  k-means  function  in  R  software 
and  the  k-means  cluster  analysis  function  in  JMP  statistical  discovery  software  from  SAS 
using  a  Type  I  error  of  less  than  0.05. 
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F. 


CONCLUSION 


In  this  chapter,  we  outlined  the  various  analytical  approaches  taken  to  analyze  our 
data  set  in  the  context  of  a  BCA.  In  reviewing  this  chapter,  the  reader  should  have 
confidence  in  the  level  of  analysis  and  research  undertaken.  The  content  should  also  expand 
the  reader’s  knowledge  of  the  techniques  used.  Evaluating  the  impacts  of  CILS— TAT  on 
individual  ships’  performance  is  a  multivariate  problem  requiring  multiple  approaches.  The 
data  mining  methods  discussed  provide  a  methodical  approach  to  analyzing  the  numerous 
variables  that  have  a  potential  impact  on  ship  performance  and,  where  possible,  isolate 
CILS— TAT  effects.  These  techniques  are  widely  accepted  as  relevant  approaches  when 
conducting  a  BCA,  and  we  felt  that  the  inclusion  of  each  was  both  appropriate  and  relevant 
to  the  research  questions.  Each  of  the  models  selected  provides  a  different  angle  from  which 
the  data  can  be  evaluated,  and  the  use  of  multiple  approaches  ensures  accuracy  and  reliability 
of  the  BCA. 
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III.  DATA  ANALYSIS 


A.  DATA  RETRIEVAL 

The  data  used  during  the  course  of  this  project  was  provided  entirely  by  NSLC.  The 
project  required  a  significant  amount  of  historical  data  for  each  of  the  18  ships  we  analyzed 
and  included  a  summary  of  all  CASREPs  submitted  from  2003  to  2011,  in-port  and 
underway  steaming  hours,  3M  registered  demand  data  from  2003  to  20 1 1  covering  more  than 
500,000  material  requests,  monthly  percentages  that  ships  were  free  from  Category  3  and 
Category  4  CASREPs,  and  supply  effectiveness  rates. 

NSLC  also  provided  a  detailed  report  for  each  of  the  18  CILS-TATs  conducted 
during  our  period  of  analysis.  These  reports  contained  the  number  of  items  recommended  to 
be  carried  as  allowances,  the  cost  of  each  item  identified,  and  the  mission  criticality  code. 

As  we  received  data,  we  grouped  and  normalized  it  to  verify  continuity  of 
information  and  determine  whether  gaps  existed  in  the  analysis  window.  Of  the  most  critical 
data  elements,  the  one  with  the  largest  amount  of  missing  data  was  the  CASREP  summary 
information.  The  project  accounts  for  a  total  of  864  months  of  operational  data  from  the  18 
ships  we  surveyed.  From  those  864  months,  there  were  42  months  in  which  the  CASREP 
data  was  either  missing  or  incomplete  and  could  not  be  provided  by  NSLC.  Because  these 
42  months  account  for  exactly  4.86%  of  our  total  sample,  the  amount  of  data  provided  was 
deemed  sufficient  for  further  analysis. 

Due  to  the  large  amount  of  supporting  data  available,  in  our  analysis,  we  were  able  to 
control  for  a  number  of  variables  that  would  otherwise  bring  the  relevancy  of  our  conclusions 
into  question.  One  variable  that  we  were  unable  to  account  for,  however,  was  the  amount  of 
funding  provided  to  the  ships  for  traditional  comprehensive  allowancing.  Because  funding 
provided  in  this  format  is  assumed  to  vary  from  ship  to  ship,  the  robustness  of  a  ship’s 
allowance  product  will  impact  both  its  supply  effectiveness  rates  and  its  incidence  of 
CASREPs. 

For  the  purposes  of  this  project,  the  ships  and  specific  time  periods  we  analyzed  are 
listed  in  Table  3.  The  ships  represent  both  the  Atlantic  and  Pacific  fleets  and  are  the  current 
members  of  the  U.S.  Navy’s  BMD  fleet. 
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Table  3. 


List  of  Ships  and  CILS— TAT  Periods  Analyzed 


Name 

UIC 

Date  Range 

MONTEREY  CG61 

"21450 

1 1/1/2003  -  1 1/30/2007 

SHILOH  CG  67 

21657 

10/1/2004-  10/31/2008 

LAKE  ERIE  CG  70 

21827 

2/1/2005  -  2/28/2009 

VELLA  GULF  CG  72 

21829 

7/1/2003  -  7/31/2007 

PORT  ROYAL  CG73 

21830 

9/1/2003  -  9/30/2007 

JOHN  PAUL  JONES  DDG  53 

21313 

8/1/2004-  8/31/2008 

STOUT  DDG  55 

21685 

12/1/2004-  12/31/2008 

JOHN  S  MCCAIN  DDG  56 

21686 

5/1/2005  -  5/31/2009 

RUSSELL  DDG  59 

"21821 

6/1/2003  -  6/30/2007 

PAUL  HAMILTON  DDG  60 

"21822 

10/1/2004-  10/31/2008 

RAMAGE  DDG  61 

21823 

7/1/2003  -  7/31/2007 

FITZGERALD  DDG  62 

"21824 

6/1/2004  -  6/30/2008 

BENFOLD  DDG  65 

"21940 

10/1/2004-  10/31/2008 

THE  SULLIVANS  DDG  68 

21942 

1 1/1/2003  -  1 1/30/2007 

MILIUS  DDG  69 

"21943 

3/1/2005  -  3/31/2009 

HOPPER  DDG  70 

"21944 

1 1/1/2003  -  1 1/30/2007 

DECATUR  DDG  73 

'21947 

8/1/2003  -  8/31/2007 

HIGGINS  DDG  76 

21950 

5/1/2003  -  5/31/2007 

B.  CILS-TAT  ANALYSIS  WINDOWS 

Throughout  the  analysis,  all  ships  were  analyzed  individually.  The  overall  data  set 
covers  periods  from  December  2003  to  January  2011.  It  is  worth  noting  that  for  the 
remainder  of  this  analysis,  a  period  represents  a  month  of  time  unless  otherwise  specified. 
For  parameters  that  are  sampled  at  frequencies  greater  then  monthly,  the  data  has  been 
averaged  or  summed  as  necessary  to  ensure  consistent  analysis  periods. 

For  the  various  ships,  CILS— TAT  was  implemented  at  different  times  during  the 
periods  from  June  2003  to  May  2005.  When  analyzing  the  individual  ships’  parameters,  we 
broke  up  the  data  sets  between  the  24  periods  before  the  implementation  of  CILS— TAT  and 
the  24  periods  after.  In  total,  the  data  set  covers  48  periods.  Table  4  identifies  the  applicable 
date  ranges  for  the  CILS— TAT  data  set  windows.  The  Before  column  in  Table  4  indicates 
the  month  and  year  that  starts  the  24  months  before  CILS— TAT;  and  similarly,  the  End 
column  indicates  the  final  month  and  year  of  the  24  months  following.  The  date  of  CILS— 
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TAT  implantation  is  indicated  by  the  Entered  column.  When  grouping  individual  ships 
before  and  after  CILS— TAT,  respective  periods  (0-48)  are  combined  accordingly,  ensuring 
consistency  in  period-to-period  analysis. 

Table  4.  CILS— TAT  Analysis  Windows 


Ship  Name 

Before 

Entered 

End 

USS  Monterey 

Nov/2003 

Nov/2003 

Nov/2007 

USS  Shiloh 

Oct/2004 

Oct/2004 

Oct/2008 

USS  Lake  Erie 

Feb/2005 

Feb/2005 

Feb/2009 

USS  Vella  Gulf 

Jul/2003 

Jul/2003 

Jul/2007 

USS  Port  Royal 

Sep/2003 

Sep/2003 

Sep/2007 

USS  John  Paul  Jones 

Aug/2004 

Aug/2004 

Aug/2008 

USS  Stout 

Dec/2004 

Dec/2004 

Dec/2008 

USS  John  S  McCain 

May/2005 

May/2005 

May/2009 

USS  Russell 

Jun/2003 

Jun/2003 

Jun/2007 

USS  Paul  Hamilton 

Oct/2004 

Oct/2004 

Oct/2008 

USS  Ramage 

Jul/2003 

Jul/2003 

Jul/2007 

USS  Fitzgerald 

Jun/2004 

Jun/2004 

Jun/2008 

USS  Benfold 

Oct/2004 

Oct/2004 

Oct/2008 

USS  The  Sullivans 

Nov/2003 

Nov/2003 

Nov/2007 

USS  Milius 

Mar/2005 

Mar/2005 

Mar/2009 

USS  Hopper 

Nov/2003 

Nov/2003 

Nov/2007 

USS  Decatur 

Aug/2003 

Aug/2003 

Aug/2007 

USS  Higgins 

May/2003 

May/2003 

May/2007 

C.  MODEL  OUTPUT  AND  ANALYSIS 
1.  Allowance  Effectiveness  Results 

In  the  analysis  of  CILS— TAT’ s  impact  on  allowance  effectiveness,  we  start  by 
looking  at  the  descriptive  statistics  for  the  before  and  after  data  sets.  Throughout  this 
evaluation,  the  expected  result  is  that  the  mean  of  the  before  data  set  is  less  than  the  mean  of 
the  after  data  set.  This  is  consistent  with  Hypothesis  2  (Equation  17)  noted  in  the 
Methodology  chapter  under  the  Regression  Analysis  section.  Looking  at  Table  5,  which 
outlines  baseline  statistics  for  the  before  and  after  data  sets,  the  entire  before  data  set  has  an 
average  allowance  effectiveness  of  66.03%,  with  values  ranging  from  100%  to  12.50%. 
Between  the  CG  and  DDG  groups,  the  average  allowance  effectiveness  is  70.08%  and 
64.47%,  respectively.  Although  these  values  appear  to  be  dissimilar,  they  are  well  within 
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one  standard  deviation  of  each  other  and  the  overall  average,  thereby  making  them 
statistically  similar.  For  the  purposes  of  comparison,  the  data  shared  between  the  two  classes 
of  ships  are  similar,  from  an  allowance  effectiveness  perspective. 


For  the  after  data,  the  overall  averages  range  from  25.30%  to  98.40%,  with  an 
average  of  71.12%.  Similarly,  CG  and  DDG  groups  average  73.97%  and  70.02%, 
respectively.  The  absolute  change  in  means  range  from  -17.57%  to  -0.25%.  Overall,  the 
average  change  in  means  for  the  data  sets  was  -5.09.  Categorically,  CGs  experienced  an 
average  change  of  -3.90,  compared  to  an  average  change  of  -5.56  experienced  by  DDGs.  To 
ensure  consistency  in  the  Welch  Two-Sample  /-test,  the  change  in  mean — as  calculated  in 
Table  5 — is  the  difference  between  the  mean  of  the  before  data  set  minus  the  mean  of  the 
after  data  set.  Therefore,  a  negative  number  in  the  Change  in  Mean  column  from  Table  5 
equates  to  an  increase  in  allowance  effectiveness  from  the  before  period  to  the  after  period. 

Table  5.  Summary  of  Changes  in  Means  due  to  CILS— TAT 


Summary  Statistics:  Allowance  Effectiveness  [CILS-TAT] 

Before  [2  years]  After  [2  Years] 


Max  Min  Average  StdDev  Variance  Max  Min  Average  StdDev  Variance 


CG 

61 

85.20 

47.10 

67.87 

11.89 

141.48 

97.60 

59.00 

80.23 

9.15 

83.78 

-12.37 

CG 

67 

93.80 

46.30 

69.71 

12.65 

160.11 

91.50 

45.90 

73.79 

10.49 

110.07 

-4.08 

CG  CG 

70 

88.30 

56.70 

69.34 

7.67 

58.86 

79.50 

41.10 

65.55 

11.03 

121.75 

3.80 

CG 

72 

86.20 

59.20 

73.80 

7.12 

50.65 

91.70 

25.30 

75.88 

13.79 

190.13 

-2.08 

CG 

73 

83.80 

46.80 

69.67 

9.88 

97.56 

93.60 

48.60 

74.42 

10.96 

120.22 

-4.75 

CG  Overall  [Avgl 

93.80 

46.30 

70.08 

10.11 

102.27 

97.60 

25.30 

73.97 

12.00 

143.99 

-3.90 

DDG 

53 

83.60 

16.90 

56.91 

15.42 

237.75 

90.70 

44.70 

71.01 

15.11 

228.29 

-14.10 

DDG 

55 

100.00 

39.70 

68.09 

14.50 

210.29 

93.40 

46.60 

66.32 

12.57 

157.94 

1.78 

DDG 

56 

86.20 

12.50 

63.70 

19.38 

375.76 

89.60 

41.10 

71.64 

12.48 

155.71 

-7.93 

All  Ships 

DDG 

59 

86.00 

37.80 

62.39 

13.71 

188.05 

85.10 

53.30 

71.87 

10.31 

106.24 

-9.48 

DDG 

60 

79.60 

40.80 

62.45 

11.02 

121.43 

89.00 

28.70 

67.10 

15.66 

245.36 

-4.65 

DDG 

61 

86.40 

43.90 

64.71 

10.27 

105.50 

95.70 

29.00 

67.52 

14.50 

210.18 

-2.81 

DDG  DDG 

62 

98.70 

38.70 

63.28 

17.58 

309.22 

92.50 

46.00 

67.13 

12.95 

167.82 

-3.86 

DDG 

65 

89.10 

15.80 

63.65 

15.23 

231.86 

92.40 

38.40 

73.69 

11.45 

131.01 

-10.04 

DDG 

68 

93.10 

47.20 

67.27 

11.40 

130.07 

82.70 

48.40 

64.30 

10.00 

100.01 

2.97 

DDG 

69 

91.50 

31.60 

66.68 

12.93 

167.24 

98.40 

69.50 

84.26 

7.96 

63.38 

-17.57 

DDG 

70 

98.90 

50.50 

72.57 

15.18 

230.42 

91.20 

43.40 

72.83 

13.17 

173.53 

-0.25 

DDG 

73 

89.30 

43.30 

67.57 

10.43 

108.88 

92.30 

45.10 

69.10 

11.99 

143.76 

-1.53 

DDG 

76 

78.40 

31.70 

58.79 

11.79 

139.07 

92.00 

29.00 

63.53 

16.04 

257.41 

-4.73 

DDG  Overall  [Avg] 

100.00 

12.50 

64.47 

14.30 

204.55 

98.40 

28.70 

70.02 

13.60 

184.83 

-5.56 

All  Sh 

ps  Ov  erall  |Avg] 

100.00 

12.50 

66.03 

13.50 

182.17 

98.40 

25.30 

71.12 

13.28 

176.26 

-5.09 

Although  all  18  ships  experienced  a  change  in  the  means  of  their  data  sets,  only  six 
ships,  highlighted  in  yellow,  had  a  change  based  on  the  Welch  Two-Sample  t-test,  which  was 
statistically  significant.  For  the  remaining  ships  that  did  not  meet  the  threshold  for  a  Type  I 
error  of  less  than  0.05,  there  was  not  enough  evidence  to  prove  that  the  variance  in  the  data 
sets  and  the  respective  changes  in  means  was  not  purely  based  on  chance.  The  intent  of  this 
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analysis  is  to  show  that  for  six  of  the  ships  in  the  sample,  CILS— TAT  was  directly 
responsible  for  the  change  in  effectiveness.  The  results  of  the  Welch  Two-Sample  /-test  are 
summarized  in  Table  6. 

Table  6.  Summary  of  Allowance  Effectiveness  and  the  Welch  Two-Sided  t-Test 


T-Test:  Allowance  Effectiveness 

Mean  of 
Before 

Mean  of 
After 

Less  Alternative  -  //.  =  Jl<)W  //,  <p„„ 

Change  in  T  Sutlatlc  p  Value  DOF  LCL 

Mean 

UCL 

CG  61 

67.87 

80.23 

-12.37 

-4.04 

0.000109 

43.17 

—  cc 

-7.22 

CG  67 

69.71 

73.79 

-4.08 

■122 

0.115242 

44.47 

—  OC 

1.56 

CG  70 

69.34 

65.55 

3.80 

1.38 

0.913032 

41.03 

—  cc 

8.41 

CG  72 

73.80 

75.88 

-2.08 

-0.66 

0.258376 

34.44 

—  cc 

3.28 

CG  73 

69.67 

74.42 

-4.75 

-1.58 

0.060877 

45.51 

-  OC 

0.31 

DDG  53 

56.91 

71.01 

-14.10 

-3.20 

0.001247 

45.98 

—  OC 

-6.70 

DDG  55 

68.09 

66.32 

1.78 

0.45 

0.673693 

45.09 

—  OC 

8.35 

DDG  56 

63.70 

71.64 

-7.93 

-1.69 

0.049876 

1927 

-  ac 

-0.01 

,  .  .  .  DDG  59 

62.39 

71.87 

-9.48 

-2.71 

0.004861 

42.70 

-  oc 

-3.59 

DDG  60 

62.45 

67.10 

-4.65 

-1.19 

0.120735 

41.29 

-  oc 

1.93 

DDG  61 

64.71 

67.52 

-2.81 

-0.78 

0.221230 

41.44 

—  oc 

3.29 

DDG  62 

63.28 

67.13 

-3.86 

-0.87 

0.195844 

42.28 

-  oc 

3.64 

DDG  65 

63.65 

73.69 

-10.04 

-2.58 

0.006679 

42.70 

—  oc 

-3.50 

DDG  68 

67.27 

64.30 

2.97 

0.96 

0.828462 

45.23 

-  oc 

8.17 

DDG  69 

66.68 

84.26 

-17.57 

-5.67 

0.000001 

38.24 

—  oc 

-12.35 

DDG  70 

72.57 

72.83 

-0.25 

-0.06 

0.475437 

45.11 

-  cc 

6.64 

DDG  73 

67.57 

69.10 

-1.53 

-0.47 

0.319392 

45.14 

—  oc 

3.92 

DDG  76 

58.79 

63.53 

-4.73 

-1,16 

0.125367 

42.24 

ac 

2.10 

The  primary  driver  that  differentiated  statistically  significant  ships  from  those  that 
were  not  statistically  significant  was  a  large  change  in  the  means  of  the  data  sets  coupled 
with  a  decrease  in  the  variance  and  standard  deviation  of  the  data  from  the  before  period  to 
the  after  period.  Figure  10,  in  the  top  graph,  shows  a  boxplot  for  each  ship’s  allowance 
effectiveness  before  (blue)  and  after  (red)  CILS— TAT.  For  each  ship,  the  dot  in  the  middle  is 
the  mean  effectiveness,  and  the  block  represents  the  range  of  +/-  one  standard  deviation  from 
the  mean.  Finally,  the  whiskers  show  the  difference  between  the  maximum  and  minimum 
allowance  effectiveness.  The  blue  line  between  the  two  data  sets  represents  the  change, 
direction  and  magnitude,  of  the  before  and  after  data.  Similarly,  plotted  in  the  bottom  section 
of  Figure  10  are  the  standard  deviations  before  and  after  for  each  ship.  This  graph  both 
identifies  the  change  in  standard  deviation  for  an  individual  ship  and  shows  these  standard 
deviations  in  the  context  of  the  overall  sample  population. 
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Variance  in  Allowance  Effectiveness  Before  to  After  CILS-TAT 

Before  CILS-TAT  -  After  CILS-TAT 


CG  CG  CG  CG  DDG  DDG  DOG  DDG  DDG  DOG  DDG  DDG  DDG  DDG  DDG  DDG  DDG 

67  70  72  73  S3  5S  S6  59  60  61  62  6S  68  69  70  73  76 

Figure  10.  Variance  in  Allowance  Effectiveness  Before  and  After  CILS-TAT 

For  five  out  of  the  six  ships,  this  is  the  exact  relationship  observed:  The  allowance 
effectiveness  increased  in  all  ships  where  the  change  was  statistically  significant.  However, 
for  DDG  53,  the  standard  deviation  of  the  data  sets  changed  only  marginally.  For  this  ship,  it 
was  the  -14.10%  change  in  allowance  effectiveness  that  was  well  above  the  upper  control 
limit  of  -6.70  that  allowed  this  value  to  be  statistically  significant.  The  key  is  that  the  ships 

that  experienced  a  statistically  significant  change  saw  less  variance  in  allowance 

effectiveness  following  the  implementation  of  CILS-TAT.  The  reduction  in  variance  from 
before  to  after  indicates  that  CILS-TAT  enabled  the  ships  to  more  tightly  and  accurately 
control  allowance  effectiveness.  Similarly,  increased  control  in  variance  resulted  in  a 
significant  increase  in  the  allowance  effectiveness  values  post  CILS-TAT — the  ships  were 
able  to  maintain  a  higher  value  of  allowance  effectiveness.  From  a  manager’s  perspective, 
we  can  conclude  that  because  of  CILS-TAT  implementation,  we  noticed  an  improved  level 
of  allowance  effectiveness  as  well  as  a  more  consistent  percentage  from  before  to  after. 

Given  the  final  results  of  the  before  and  after  data,  CILS-TAT  had  a  direct  and 
positive  impact  on  allowance  effectiveness  for  six  out  of  18  ships.  Of  the  12  remaining  ships, 
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nine  saw  a  positive  change  in  allowance  effectiveness,  although  this  change  is  not 
statistically  significant  and  thus  cannot  be  attributed  to  CILS— TAT. 


2.  Casualty  Report  Frequency  and  Percentage  of  Time  Free  Results 

The  CASREP  data  used  represents  the  daily  count  for  Categories  2,  3,  and  4.  For 
review,  CASREPs  reflect  broken  or  inoperable  equipment  onboard  U.S.  Navy  ships.  In  order 
to  aggregate  the  data,  the  CASREPs  were  totaled  over  the  monthly  period  to  be  consistent 
with  the  periodicity  of  allowance  effectiveness  data.  However,  for  the  purposes  of  CASREP 
frequency  analysis,  the  various  categories  of  CASREPs  were  summed  together  with  C3  and 
C4  CASREPs,  representing  less  than  20%  of  all  CASREPs. 


Table  7  shows  that  seven  out  of  the  18  ships  resulted  in  a  positive  change  in  mean 
(before  minus  after).  Of  these  seven  ships,  only  two  of  the  ships  experienced  a  change  with  a 
Type  I  error  less  than  0.05.  None  of  the  ships  that  experienced  a  statistically  significant 
change  in  allowance  effectiveness  saw  a  similar  result  for  CASREPs.  Overall,  the  average 
change  in  total  CASREPs  was  -0.5. 

Table  7.  t-Test  Results  for  Total  Casualty  Reports 


ICILS-TATl 


Hull  Number 

Mean  of 

Before 

Mean  of 
After 

Change  In 
Mean 

T  Statistic 

DOF 

P  Value 

LCL 

UCL 

CG  61 

10.52 

8.43 

2.09 

0.98 

40.54 

0.166151 

-1.49 

F  00 

Allowance 

Effectiveness 

DDG53 

6.24 

7.82 

-1.58 

-0.82 

28.33 

0.790945 

-4.86 

F  OD 

DDG  56 

3.83 

6.9S 

-3.12 

-1.93 

25.83 

0.967970 

-5.87 

+  00 

P  Value  <  0.05 

DDGS9 

11.41 

5.00 

6.41 

1.69 

17.27 

0.054497 

-0.18 

F  DO 

DDG  65 

6.00 

5.95 

0.05 

0.03 

39.64 

0.488102 

-2.76 

F  DO 

DDG  69 

3.89 

4.20 

-0.31 

-0.25 

34.78 

0.597668 

-2.42 

4-  QO 

P  Value  <  0.05  [AvrI 

6.98 

6.39 

0.59 

-0.05 

31.06 

0.510889 

-2.93 

F  QO  1 

CG  67 

6.58 

7.30 

-0.73 

-0.37 

39.24 

0.643317 

-4.03 

F  00 

CG  70 

5.68 

8.17 

-2.49 

-1.26 

36.14 

0.891559 

-5.83 

F  00 

CG  72 

9.68 

11.13 

-1.44 

-0.53 

42.64 

0.700913 

-6.01 

F  00 

CG  73 

7.95 

12.21 

-4.26 

-1.61 

39.14 

0.941882 

-8.72 

F  00 

Allowance 

Effectiveness 

DDG  55 

10.85 

9.32 

1.53 

0.61 

32.28 

0.272666 

-2.71 

F  00 

DDG  60 

8.95 

14.96 

-6.01 

-1.71 

30.20 

0.951378 

-11.96 

F  00 

DDG  61 

P  Value  >  0.05 

8.35 

6.32 

2.03 

0.88 

40.09 

0.191224 

-1.84 

F  00 

DDG  62 

4.68 

6.61 

-1.92 

-1.78 

39.97 

0.958962 

-3.74 

F  oo 

DDG  68 

8.05 

11.57 

-3.52 

-1.86 

38.72 

0.964697 

-6.70 

F  oo 

DDG  70 

3.76 

6.79 

-3.03 

-1.32 

29.72 

0.901880 

-6.91 

F  oo 

DDG  73 

4.65 

3.36 

1.29 

1.55 

39.87 

0.065117 

-0.12 

F  oo 

DDG  76 

9.25 

4.36 

4.89 

2.23 

30.90 

0.016559 

1.17 

F  oo 

P  Value  >  0.05  [Avgj 

7.37 

8.51 

-1.14 

-0.43 

36.58 

0.625013 

-4.78 

»  00 

1  A1  Shlp«  [Avg| _ 

7.24 

7,80 

-0.56 

-0.30 

34.74 

0.586972 

-4.17 

F  00 

The  POTF  metric,  which  is  calculated  as  a  quarterly  percentage,  represents  the 
amount  of  time  a  ship  operates  without  a  Category  3  or  4  CASREP.  Of  the  18  ships 
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evaluated,  seven  had  a  negative  change  in  POTF.  Table  8  shows  that  the  mean  of  the  after 
data  is  greater  than  the  mean  of  the  before  data  with  a  maximum  change  of  -29.7 1 .  Although 
these  ships  experienced  a  change  in  POTF,  only  three  of  the  ships  were  determined  to  be 
statistically  significant.  It  is  important  to  note,  however,  that  POTF  is  reported  on  a  quarterly 
basis,  and  for  the  48  months  analyzed,  there  are  only  16  data  points,  which  is  considered  a 
small  sample  for  analysis. 

Table  8.  t-Test  Results  for  Percentage  of  Time  Free  From  Casualty  Report 


T-Test:  Percentage  of  Time  Free  from  CASREP  (POTF)  [CILS-TAT] 
Hull  Means  of  Means  of  Change  in 

Number  Before  After  Means 

T  Statistic 

DOF 

P  Value 

LCL 

UCL 

CG  61 

77.60 

88.63 

-11.03 

-130 

10.04 

0.128694 

—  00 

5.61 

DDG  53 

63.63 

53.00 

10.63 

0.65 

13.03 

0.737322 

—  00 

39.4S 

.Allowance  DDG  56 

Effectiveness  __  _  __ 

70.00 

72.50 

-2.50 

-0.16 

10.73 

0.436541 

—  ao 

25.00 

P  Value  <0  05  DDG  59 

100.00 

40.13 

59.88 

5.49 

7.00 

0.999542 

—  ao 

80.53 

DDG  65 

87.29 

54.00 

33.29 

2.97 

9.02 

0.992218 

—  ao 

53.80 

DDG  69 

74.25 

64.50 

9.75 

0.67 

13.81 

0.744204 

—  ao 

35.26 

P  Value  <  0.05  [Avg| 

78.79 

62.13 

16.67 

1.40 

10.61 

0.673087 

—  ao 

39.94 

CG  67 

42.29 

72.00 

-29.71 

-1.88 

10.78 

0.0433S7 

—  ao 

-1.35 

CG  70 

78.25 

37.00 

41.25 

2.44 

13.99 

0.985796 

—  ao 

70.99 

CG  72 

96.67 

72.75 

23.92 

2.34 

8.35 

0.976894 

-  ao 

42.83 

CG  73 

71.00 

48.50 

22.50 

0.99 

7.60 

0.823758 

—  ao 

65.04 

DDG  55 

9.25 

41.63 

-3238 

-2.02 

12.45 

0.032608 

—  ao 

-3.92 

.Allowance  DDG  60 

Effectiveness  _ 

96.14 

73.75 

22.39 

332 

8.84 

0.994615 

—  ao 

35.17 

P  Value  >  0.05  DDG  61 

72.00 

80.63 

-8.63 

-0.53 

6.12 

0306526 

—  ao 

22.73 

DDG  62 

67.71 

60.63 

7.09 

0.43 

12.12 

0.663242 

—  ao 

36.33 

DDG  68 

80.80 

70.75 

10.05 

0.58 

10.79 

0.713115 

—  ao 

41.22 

DDG  70 

89.60 

74.63 

14.98 

1.50 

10.60 

0.919007 

-  ao 

32.93 

DDG  73 

68.50 

82.13 

-13.63 

-0.66 

3.94 

0.273569 

—  ao 

30.72 

DDG  76 

59.33 

78.38 

-19.04 

-1.93 

8.88 

0.042774 

—  ao 

-0.97 

P  Value  <  0.05  [A\g| 

69.30 

66.06 

3.23 

037 

9.54 

0.564605 

-  ao 

30.98 

All  Ships  [Avg] 

72.46 

64.75 

7.71 

0.72 

9.90 

0.600766 

—  ao 

33.97 

Overall,  based  on  the  minimal  changes  in  CASREP  frequency  and  the  lack  of  a 
statistically  significant  change  of  POTF,  CILS-TAT  appears  to  have  had  a  negligible  impact 
on  the  number  of  CASREPs  experienced  and  the  amount  of  time  CASREPs  are  open. 
Additionally,  ships  that  saw  an  increase  in  allowance  effectiveness  as  a  result  of  CILS-TAT 
did  not  see  a  reduction  in  CASREP  frequency  or  marked  improvement  in  POTF.  The 
inclusive  statistical  relationship  could  be  explained  by  an  assumption  that  the  processing  of 
CASREPs  onboard  ships  is  often  riddled  with  variable  subjectivity  introduced  by  the 
controllers  of  the  process. 

3.  Impact  of  CILS-TAT  Spending 

To  measure  the  impact  of  the  dollars  spent  on  CILS-TAT,  and  the  respective  changes 
in  allowance  effectiveness,  we  turn  to  the  amount  of  money  obligated.  Table  9  shows  a 
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breakdown  of  the  actual  dollar  amounts  obligated  in  the  year  that  CILS— TAT  was 
implemented.  These  values  have  been  adjusted  to  constant  fiscal  year  (FY)  2003  (base  year) 
dollars  using  the  OPN  inflation  category  found  in  the  2012  Joint  Inflation  Calculator. 
FY2003  was  chosen  as  the  base  year  in  conjunction  with  the  beginning  of  the  first  CILS— 
TAT  analysis  periods. 


Table  9.  CILS— TAT  Obligation  Amounts 


Hull 

Number 

Year 

Obligated 

Inflation 

Factor 

Actual 

Obligated 

Value 

Actual  Obligated 
Value  ($FY2003) 

CG  61 

2005 

0.9537 

$234,668.92 

$223,801.14 

CG  67 

2006 

0.9250 

$161,646.56 

$149,525.24 

CG  70 

2007 

0.9007 

$184,329.58 

$166,024.67 

CG  72 

2005 

0.9537 

$229,774.69 

$219,133.56 

CG  73 

2005 

0.9537 

$292,576.74 

$279,027.18 

DDG  53 

2006 

0.9250 

$837.01 

$774.25 

DDG  55 

2006 

0.9250 

$751.57 

$695.21 

DDG  56 

2007 

0.9007 

$92,066.71 

$82,924.00 

DDG  59 

2005 

0.9537 

$201,782.00 

$192,437.25 

DDG  60 

2006 

0.9250 

$119,360.84 

$110,410.38 

DDG  61 

2005 

0.9537 

$182,834.45 

$174,367.18 

DDG  62 

2006 

0.9250 

$318,133.93 

$294,278.16 

DDG  65 

2006 

0.9250 

$227,216.30 

$210,178.13 

DDG  68 

2005 

0.9537 

$62,657.90 

$59,756.14 

DDG  69 

2007 

0.9007 

$86,102.12 

$77,551.72 

DDG  70 

2005 

0.9537 

$159,333.84 

$151,954.91 

DDG  73 

2005 

0.9537 

$148,749.17 

$141,860.43 

DDG  76 

2005 

0.9537 

$229,568.54 

$218,936.96 

Total 

$2,932,390.87 

$2,753,636.52 

For  purposes  of  comparison,  Figure  1 1  brings  together  the  allowance  effectiveness  t- 
test  results  and  the  dollar  amounts  obligated  after  CILS— TAT.  FY2003  obligated  values 
were  plotted  against  the  observed  change  in  allowance  effectiveness  means  and  the 
respective  /-test  p-value.  Ships  that  were  identified  as  statistically  significant  are  annotated 
by  red  circles. 
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Impact  of  Dollars  Spent  on  Change  in  Allowance  Effectiveness 


DOG 


•DOG  « 

o 

1 

T  T 

| 

o 

•  Ilf  11  ^  300000 

o 

• 

• 

t  1 

• 

» 

^ioooo,  xV 

°  J 

<0  -IS  -10  4  0  § 

Change  in  Effectiveness  Means 

Change  in  Means 


P  Value 


P  Value 


Change  in  Means 


Figure  11.  Explaining  the  Relationship  Between  Amounts  Obligated  and  the 
Significance  of  the  Change  in  Allowance  Effectiveness  for  Each  Ship 

The  top  graph  in  Figure  1 1  shows  how  the  three  variables  interact.  The  bottom  charts 
are  two-dimensional  projections  of  the  various  variables  from  the  top  chart.  Starting  with  the 
amount  obligated  versus  the  change  in  means,  while  a  strong  relationship  between  these  two 
does  not  appear  to  exist,  using  cluster  analysis  and  the  k-means  function  in  R  Statistics 
software,  the  ships  were  grouped  into  three  clusters  with  similar  characteristics.  Fifteen 
parameters  were  used  as  variables  in  the  cluster  analysis,  including  obligated  amount,  p- 
values,  and  changes  in  means  for  the  various  source  codes.  These  variables  were  used  in  an 
effort  to  find  those  parameters  that  differentiate  the  ships  that  experienced  a  statistically 
significant  difference  in  means  and  those  that  did  not.  A  grouping  of  three  clusters  was  used 
because  it  naturally  split  the  /-test  p-values  into  three  levels  of  Type  I  errors,  with  one  group 
centered  at  approximately  0.05. 
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Based  on  this  analysis,  Cluster  1  (blue  ellipse)  encompasses  all  ships  with  a  Type  I 
error  approximately  less  than  0.05.  Cluster  1  is  centered  on  a  p-value  of  0.03,  a  change  in 
means  of -10.11%,  and  an  obligated  amount  of  $147,138.  Within  this  cluster,  the  ships  can 
be  split  again  into  two  groups  based  on  the  amount  that  was  obligated.  In  the  first  set 
(bottom  left),  DDG  53  and  DDG  69  achieved  a  significant  change  in  means  with  an  obligated 
amount  almost  half  of  the  other  statistically  significant  ships — less  than  $100,000.  Included 
in  this  grouping,  DDG  56  did  not  see  as  large  of  a  change  in  means  for  a  similar  amount 
obligated,  but  the  change  was  enough  to  be  statistically  significant. 

In  the  second  grouping  in  the  first  cluster  (top  left)  are  CG  61,  DDG  59,  and  DDG  65, 
with  amounts  obligated  ranging  from  $174,000  to  $210,000.  They  achieved  a  smaller  change 
in  the  means  of  allowance  effectiveness,  with  an  average  change  of  -10.63%.  The  other  two 
clusters  of  ships,  Cluster  2  (red  ellipse)  and  Cluster  3  (green  ellipse),  encompass  the 
remaining  ships  whose  change  in  allowance  effectiveness  was  not  below  the  p-value 
threshold  of  less  than  0.5.  The  important  takeaway  from  this  analysis  is  that  the  amount  of 
money  obligated  appears  to  be  independent  of  a  change  in  allowance  effectiveness.  This  is 
true  in  terms  of  both  magnitude  of  the  change  and  its  statistical  significance.  For  roughly  the 
same  amount  obligated,  Clusters  1  (blue  ellipse)  and  2  (red  ellipse)  achieved  vastly  different 
changes  in  means  and  statistical  significance. 

We  also  used  cluster  analysis  to  examine  individual  source  codes.  There  are  two  for 
Cluster  1,  source  codes  A  and  C,  for  which  almost  all  ships  experienced  a  significant  change. 
Figure  12  shows  the  relative  magnitude  of  the  change  in  means  and  their  statistical 
significance  for  the  three  clusters.  The  values  in  Figure  12  have  been  normalized  to  allow  for 
better  comparison  between  variables  with  dissimilar  scales.  It  is  important  to  note  that  in 
terms  of  source  codes  (see  Figure  4)  A,  C,  D,  and  F,  the  expected  change  in  means  is 
negative,  and  for  source  codes  G  and  J,  the  expected  change  is  positive.  This  indicates  that 
for  the  first  four  source  codes  of  A,  C,  D,  and  F,  the  lower  the  value  for  change  in  means,  the 
better;  and  for  the  final  two  source  codes  of  G  and  J,  the  larger  the  value,  the  better.  For  the 
ships  that  are  statistically  significant,  Cluster  1,  it  is  clear  that  these  ships  experienced  the 
largest  changes  in  four  out  of  six  source  codes;  however,  only  source  codes  A  and  C 
experienced  Type  I  errors  less  than  0.05. 
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Figure  12.  Source  Codes  That  Drive  Allowance  Effectiveness  to  Be  Statistically 

Significant  Using  Cluster  Analysis 

Table  10,  which  shows  the  results  of  t-test  analysis  for  the  various  source  codes  in 
conjunction  with  Figure  13  in  terms  of  its  change,  shows  that  source  code  C  had  the  lowest  p- 
value  both  individually  and  when  averaged  across  the  six  statistically  significant  ships. 
However,  in  terms  of  numerical  magnitude  and  the  parameters’  relative  ability  to  change  the 
allowance  effectiveness  equation,  the  primary  parameter  of  concern  is  source  code  A.  This 
can  be  seen  in  the  ellipse  for  change  in  source  code  A,  which  is  centered  on  -57.73,  while  the 
ellipse  for  source  code  C  is  centered  on  -12.18.  In  summary,  we  found  that  source  code  A 
had  a  greater  ability  to  impact  the  change  in  allowance  effectiveness  than  any  other  source 
code 


In  keeping  with  this  theme,  overall  changes  in  source  code  G  are  not  statistically 
significant  within  a  95%  confidence  interval.  This  can  be  explained  by  looking  at  the  t-test 
results  for  source  code  G.  Looking  at  the  p-values  for  the  greater-than  and  less-than 
hypothesis,  there  does  not  appear  to  be  a  clear  direction  among  all  of  the  six  ships  with 
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statistically  significant  change  nor  among  all  ships  for  the  change  in  source  code  G;  out  of  all 
of  the  ships,  33%  were  statistically  significant  with  a  greater-than  hypothesis  and  17%  with  a 
less-than  hypothesis.  In  terms  of  source  code  J,  the  ships  in  Cluster  1  fell  between  the  other 
clusters.  Similarly,  this  is  explained  by  the  ambiguous  direction  of  the  change  in  means 
among  all  ships;  for  the  greater-than  hypothesis,  17%  of  the  ships  experienced  a  Type  I  error 
less  than  0.05,  and  likewise,  under  the  less-than  hypothesis,  17%  of  the  ships  experienced  a 
Type  I  error  less  than  0.05.  Essentially,  the  results  of  the  G  and  J  source  codes  are  not 
consistent  and  not  statistically  significant.  The  data  is  split  unevenly  in  directionality.  Early 
on,  CILS— TAT  affected  A  and  C  but  did  not  affect  the  G  and  J  source  codes.  According  to 
the  framework  of  source  codes,  one  would  expect  some  correlation  between  A/C  and  G/J. 
As  stated  previously,  the  data  analysis  does  not  reveal  any  correlation  between  the  four 
source  codes  of  A,  C,  G,  and  J. 

These  changes  are  consistent  with  the  expected  results  of  CILS— TAT  implementation 
because  the  technique  is  designed  to  increase  the  frequency  of  source  codes  A  and  C.  If 
demands  received  in  the  after  period  are  identical  to  demands  received  in  the  before  period, 
then  one  would  expect  the  occurrence  of  source  codes  A  and  C  to  increase  while  G  and  J 
decreased  in  a  perfectly  inverse  relationship.  In  reality,  however,  there  are  other  factors  that 
influence  the  frequency  of  G  and  J  that  cannot  be  completely  mitigated  by  CILS— TAT  (e.g., 
poor  configuration  management  practices).  This  proves  that,  independent  of  the  amount  of 
money  obligated,  in  those  ships  that  achieved  a  statistically  significant  change  in  allowance 
effectiveness,  CILS— TAT  was  able  to  accurately  target  source  codes  A  and  C.  This  implies 
that  the  true  measure  of  CILS— TAT’ s  impact  on  allowance  effectiveness  can  be  quantified 
simply  by  focusing  analysis  on  changes  in  source  codes  A  and  C. 
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Table  10.  Analysis  of  Source  Codes  for  Allowance  Effectiveness  Calculation 


T-Test  IP  Value]:  Allowance  Effectiveness  Source  Codes  Analy  sis  |CTLS-TAT  Period] 

Source  Code  C  Source  Code  D  Source  Code  P  Source  Code  G  Source  Code  J 


Hull 

Number 

Greater 

Less 

Greater 

Lest 

Greater 

Less 

Greater 

Leu 

Greater 

Leu 

Greater 

Leu 

Greater 

Leu 

CG  61 

0.9999 

0.0001 

0.9998 

0.0002 

0.9984 

0.0016 

0.9306 

0.0694 

0.99% 

0.0004 

0.5157 

0.4843 

0.2295 

0.7705 

DDG  53 

Allow  tnce 

Fffm1vr.ni  00056 

0.9988 

0.0012 

0.9994 

0.0006 

0.9997 

0.0003 

0.9951 

0.0049 

0.6573 

0.3427 

0.0036 

0.9964 

0.8045 

0.1955 

0.9501 

0.0499 

0.9883 

0.0117 

1. 0000 

0.0000 

1.0000 

0.0000 

0.6505 

0.3495 

0.0546 

0.9454 

0.9752 

0.0248 

“v.l"  DDG'‘' 

0.9951 

0.0049 

0.9999 

0.0001 

0.9697 

0.0303 

0.7836 

0.2164 

0.4236 

0.5764 

0.9547 

0.0453 

0.6293 

0.3707 

<0.01  DDG65 

0.9933 

0.0067 

0.9927 

0.0073 

1.0000 

0.0000 

0.0031 

0.9969 

0.0773 

0.9227 

0.0015 

0.9985 

0.2009 

0.7991 

DDG  69 

1.0000 

0.0000 

0.8328 

0.1672 

1.0000 

0.0000 

0.9991 

0.0009 

0.0348 

0.9652 

0.0002 

0.9998 

0.0012 

0.9988 

P  Value  <  0.65  (Average) 

0.9895 

0.0105 

0.9688 

0.0312 

0.9946 

0.0054 

0.7852 

0.2148 

0.4738 

0.5262 

0.2550 

0.7450 

0.4734 

0.5266 

CG  67 

0.8848 

0.1152 

0.4458 

0.5542 

1.0000 

0.0000 

0.0059 

0.9941 

0.9928 

0.0072 

0.0133 

0.9867 

0.1142 

0.8858 

CG  70 

0.0870 

0.9130 

0.6767 

0.3233 

0.6379 

0.3621 

0.9818 

0.0182 

0.9958 

0.0042 

0.9230 

0.0770 

0.9925 

0.0075 

CG  72 

0.7416 

0-2584 

0.9361 

0.0639 

0.0000 

UNW 

0.8464 

0.1536 

0.0841 

0.9159 

0.9038 

0.0962 

0.3629 

0.6371 

CG  73 

0.9391 

0.0609 

0.9982 

0.0018 

0.9886 

0.0114 

0.7845 

0.2155 

0.7052 

0.2948 

0.7088 

0.2912 

0.7205 

0.2795 

Allowance  DDG  55 

0.3263 

0.6737 

0.9453 

0.0547 

0.0146 

0.9854 

0.9935 

0.0065 

0.9979 

0.0021 

0.7838 

0.2162 

0.9932 

0.006R 

Effectiveness  DDG  60 

0.8793 

0.1207 

0.8238 

0.1762 

1.0000 

0.0000 

0.9614 

0.0386 

0.9986 

0.0014 

0.0297 

0.9703 

0.8825 

0.1175 

P  Value  DDG  61 

0.7788 

0.2212 

0.0087 

0.9913 

0.4629 

0.5371 

0.9959 

0.0041 

0.0945 

0.9055 

0.5556 

0.4444 

0.0114 

0.9886 

>8.05  DDG  62 

0.8042 

0.1958 

0.7777 

02223 

0.3254 

0.6746 

0J124 

0.6876 

0.9674 

0.0326 

0.0007 

o 

08388 

0.1612 

DDG  68 

0.1715 

0.8285 

0.4130 

0.5870 

0.6932 

0.3068 

0.2228 

0.7772 

0.0001 

0.9999 

0.1321 

0.8679 

07680 

0.2320 

DDG  70 

0.5246 

0.4754 

0.9993 

0.0007 

0.4345 

0.5655 

0.9936 

0.0064 

IVTSM 

0.0149 

1.0000 

0.0000 

07420 

0.2380 

DDG  73 

0.6806 

0.3 194 

022597 

0.7403 

0.6250 

0.3750 

0.0443 

0.9557 

0.9322 

0.0678 

0.5000 

0.5000 

0.0714 

0.9286 

DDG  76 

0.8746 

0.1254 

0.4467 

0.5533 

0.1361 

0.8639 

0.2098 

0.7902 

0.0003 

0.9997 

0.9999 

0.0001 

0.0012 

0.998  B 

P  Value  >  0.05  (Average) 

0.6410 

0J590 

0.6442 

0J558 

0.5265 

0.4735 

0.6127 

0.3873 

0.6462 

0J538 

0.5459 

0.4541 

0.5415 

0.4585 

%  of  Ships  Statistically 

Significant  Ships  with 

OK 

100% 

OK 

83K 

0% 

100% 

17K 

50K 

17% 

17K 

50% 

17K 

17K 

17K 

P  Val  <  0.05 
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0% 

OK 
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17K 
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17K 

42K 
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P  Val  <0.05 

0% 
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6% 

39% 

11% 
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17% 
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Figure  13.  Statistical  Significance  of  Change  in  Allowance  Effectiveness  (Before  to 

After  CILS-TAT) 

The  graphs  in  Figure  13  depict  the  relationship  between  the  observed  changes  in  data 
set  means  and  their  statistical  significance  relative  to  the  p-value  and  the  95%  upper 
confidence  level  (UCL).  The  UCL  would  be  defined  in  this  case  as  a  type  of  interval 
estimate  of  a  population  parameter  and  is  used  to  indicate  the  reliability  of  an  estimate. 

These  two  graphs  show  that  of  the  data  sets,  which  did  not  show  statistical 
significance,  there  are  some  ships  (Cluster  2)  relatively  close  to  achieving  statistical 
significance.  Relating  Figure  13  back  to  Table  10,  if  the  confidence  interval  was  increased  to 
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99%,  the  number  of  statistically  significant  ships  decreases  to  five  (one  CG  and  four  DDGs). 
However,  in  order  to  increase  the  number  of  statistically  significant  ships  to  90%  of  all  ships 
(~16  ships),  the  confidence  interval  must  be  changed  to  30%,  which  is  well  outside  of 
acceptable  limits.  Again,  statistically  significant  means  that  at  a  95%  confidence  interval 
(Type  I  error  <  0.05),  there  is  enough  evidence,  based  on  the  change  in  means  and  the 
variance  in  the  two  data  sets,  to  say  that  there  was  a  change.  Note  that  there  are  three  ships — 
CG  70,  DDG  55,  and  DDG  68 — that  experienced  a  positive  change  in  means,  which  is 
opposite  to  the  expected  output.  Herein,  this  shows  that  while  there  are  some  ships  on  the 
cusp  of  achieving  statistical  significance,  only  marginal  losses  or  gains  are  made  by 
increasing  or  decreasing  the  confidence  interval — the  ships  that  achieved  statistical 
significance  were  well  inside  the  95%  confidence  interval. 

In  conclusion,  during  the  periods  when  CILS— TAT  was  implemented,  there  were  six 
ships  that  experienced  a  significant  enough  change  in  allowance  effectiveness  to  result  in  a 
Type  I  error  less  than  0.05.  Although  the  amounts  obligated  for  each  ship  varied 
significantly,  there  appears  to  be  no  relationship  between  the  amount  obligated  and  the 
resulting  change  in  allowance  effectiveness.  This  result  is  consistent  for  both  ships  that 
experienced  a  statistically  significant  change  and  those  that  did  not.  In  terms  of  the 
parameters  used  to  calculate  allowance  effectiveness,  CILS— TAT  predominately  impacted 
source  codes  A  and  C  with  some  change  to  codes  G  and  J,  however,  not  in  a  statistically 
significant  manner. 

4.  Brownout  Period  Analysis 

Data  analyzed  during  the  brownout  period  covers  December  2009  to  January  2011. 
In  order  to  maintain  uniformity  of  CILS— TAT  effectiveness  for  those  ships  whose  24-month 
window  continued  past  December  2009,  their  brownout  analysis  window  is  adjusted.  For 
example,  USS  John  S  McCain's  brownout  period  started  as  of  June  2009,  because  the  24 
months  following  CILS— TAT  implementation  did  not  complete  until  June  2009.  This 
ensures  that  all  ships’  data  are  given  the  full  24-month  window  following  CILS— TAT 
implementation. 
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During  the  brownout  period,  there  was  a  considerable  decrease  in  allowance 
effectiveness  across  the  board.  Looking  at  the  two  data  sets  within  the  before  data,  the 
reported  allowance  effectiveness  ranges  from  a  minimum  of  25.30%  to  a  maximum  of  98.40% 
with  an  overall  average  of  71.12%  and  a  standard  deviation  of  13.28%.  In  terms  of  ships 
grouped  by  class,  there  is  not  an  appreciable  difference  in  the  before  data  sets.  Among  the 
after  data  sets,  allowance  effectiveness  ranges  from  a  minimum  of  0.00%  to  a  maximum  of 
100.00%  with  an  overall  average  of  66.35%  and  a  standard  deviation  of  13.96.  Table  11 
summarizes  the  changes  in  the  means  data  sets  from  before  to  after.  To  be  consistent  for 
comparison  purposes  with  the  change  in  means  calculated  in  the  Welch  Two-Sided  t-test,  the 
change  in  means  calculation  is  based  on  the  before  data  set  mean  minus  the  after  data  set 
mean. 


Table  11.  Summary  of  Allowance  Effectiveness  (Before  to  After  Brownout  Period) 


Summiry  Statistic*:  Allowance  Effectiveness  [Brownout  PeHodj 


Hull 

Number 

Max 

Min 

Before 

Average 

StdDev 

Variance 

Max 

Min 

After 

Average 

StdDev 

Variance 

Change  In 
Mean 

CG 

61 

97.60 

59.00 

80.23 

9.15 

83.78 

90.60 

40.00 

67.31 

12.66 

160.17 

12.93 

CG 

67 

91.50 

45.90 

73.79 

10.49 

110.07 

91.00 

41.10 

67.94 

12.73 

161.95 

5.85 

CG  CG 

70 

79.50 

41.10 

65.55 

11.03 

121.75 

89.10 

39.80 

57.64 

12-21 

148.98 

7.90 

CG 

72 

91.70 

25.30 

75.88 

13.79 

190.13 

93.00 

48.70 

73.57 

10.46 

109.49 

2.31 

CG 

73 

93.60 

48.60 

74.42 

10.96 

120.22 

89.80 

46.20 

66.52 

10.36 

107.25 

7.90 

CG  Overall  [Avg| 

97.60 

25.30 

73.97 

12.00 

143.99 

93.00 

39.80 

67.25 

12.49 

156.05 

6.72 

DDG 

53 

90.70 

44.70 

71.01 

15.11 

228.29 

100.00 

0.00 

62.41 

21.39 

457.74 

8.60 

DDG 

55 

93.40 

46.60 

66.32 

12.57 

157.94 

87.90 

19.20 

63.72 

17.20 

295.82 

2.60 

DDG 

56 

89.60 

41.10 

71.64 

12.48 

155.71 

80.90 

31.40 

65.45 

13.14 

172.64 

6.18 

DDG 

59 

85.10 

53.30 

71.87 

10.31 

106.24 

90.60 

35.20 

73.09 

11.44 

130.77 

-122  | 

DDG 

60 

89.00 

28.70 

67.10 

15.66 

145  36 

83.90 

25.90 

53.81 

11.43 

130.70 

13.29 

DDG 

61 

95.70 

29.00 

67.52 

14.50 

210.18 

92.60 

26.70 

63.15 

12.43 

154.60 

4.37 

DDG  DDG 

62 

92.50 

46.00 

67.13 

12.95 

167.82 

92.70 

34.50 

66.72 

13.86 

192.05 

0.42 

DDG 

65 

92.40 

38.40 

73.69 

11.45 

131.01 

100.00 

25.00 

60.61 

14.79 

218.82 

13.08 

DDG 

68 

82.70 

48.40 

64.30 

10.00 

100.01 

90.90 

12.00 

65.31 

14.02 

196.45 

-1.01  1 

DDG 

69 

98.40 

69.50 

84.26 

7.96 

63.38 

94.90 

55.60 

73.34 

11.09 

123.08 

10.92 

DDG 

70 

91.20 

43.40 

72  83 

13.17 

173.53 

88.70 

29.60 

69.28 

12.87 

165.69 

3.55 

DDG 

73 

92.30 

45.10 

69.10 

11.99 

143.76 

95.30 

36.00 

71.49 

11.24 

126.39 

-2J9 

DDG 

76 

92.00 

29.00 

63.53 

16.04 

257.41 

88.10 

32.50 

65.85 

12.15 

147.52 

-2.33 

DDG  Overall  1  Aval 

98.40 

28.70 

70.02 

13.60 

184.83 

100.00 

0.00 

66.00 

14.49 

209.87 

4.03 

Ships  Overall 

98.40 

25.30 

71.12 

13.28 

176.26 

100.00 

0.00 

66J5 

13.96 

194.74 

4.77 

In  terms  of  statistical  significance  for  the  brownout  period,  the  expected  result  was 
that  the  mean  of  the  before  data  set  would  be  greater  than  the  mean  of  the  after  data  set. 
Based  on  this  expected  movement,  the  greater  hypothesis  was  used  as  the  base  for  analyzing 
the  brownout  period.  Again  using  the  Welch  Two-Sided  /-test,  10  ships  (four  CGs  and  six 
DDGs)  saw  a  statistically  significant  change  in  the  means  of  the  before  and  after  data  sets. 
On  average,  all  of  the  ships  experienced  a  change  of  4.77.  When  grouped  together,  CGs  and 
DDGs  experienced  an  average  change  of  6.72  and  4.03,  respectively,  in  the  data  sets. 
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In  stark  contrast  to  the  change  experienced  during  the  CILS— TAT  period,  during  the 
brownout  period,  at  a  confidence  interval  of  99%  (Type  1  error  <  0.01),  there  is  enough 
evidence  to  prove  that  six  ships  experienced  the  expected  change  in  allowance  effectiveness. 
There  are,  however,  four  ships  that  experienced  a  small,  negative  change  in  mean,  resulting 
in  very  low  statistical  significance.  For  the  brownout  period,  in  order  to  capture  90%  of  the 
ships  (~16  ships)  being  statistically  significant,  the  confidence  interval  would  need  to  shift  to 
60%.  Table  12  summarizes  the  /-test  results. 


Table  12.  /-Test  Summary  of  Allowance  Effectiveness 


Allowance  Effectiveness  [Brownout]  Greater  Alternative-  H  r  =  7/  ,An  //, :  p„,./ivv  >  {i 

Mean  of  Mean  of  Change  In  _  „  _ _  _  _ _ 

„  r  „„„  T  Statistic  P  Value  DOF  I.CL  UCL 


CG  61 

80.23 

66.14 

14.10 

5.02 

0.000003 

57.48 

9.41 

+  00 

CG  67 

73.79 

67.99 

5.80 

1.97 

0.027090 

54.12 

0.87 

+  00 

CG  70 

65.55 

57.64 

7.90 

2.57 

0.006508 

52.56 

2.76 

+  00 

CG  72 

75.88 

72.16 

3.72 

1.15 

0.128849 

37.77 

-1.74 

+  CO 

CG  73 

74.42 

63.49 

10.93 

4.29 

0.000063 

36.60 

6.63 

+  00 

DDG  53 

71.01 

61.63 

9.38 

1.94 

0.028761 

57.98 

129 

+  00 

DDG55 

66.32 

63.72 

2.60 

0.68 

0.251142 

57.44 

-3.83 

+a> 

DDG  56 

71.64 

65.45 

6.18 

1.78 

0.040484 

50.76 

0.37 

+  00 

DDG  59 

Individual 

71.87 

73.48 

-1.61 

-0.59 

0.722637 

49.29 

•6.16 

+  QO 

DDG  60 

67.10 

54.01 

13.09 

3.51 

0.000575 

39.31 

6.80 

+  00 

DDG  61 

67.52 

61.15 

6.37 

1.77 

0.042202 

44.04 

0.31 

+  00 

DDG  62 

67.13 

67.18 

-0.05 

-0.01 

0.505766 

50.14 

-5.82 

+  00 

DDG  65 

73.69 

60.49 

13.20 

3.89 

0.000134 

56.59 

7.52 

+  00 

DDG  68 

64.30 

62.95 

1.35 

0.43 

0.335418 

57.90 

-3.94 

+  00 

DDG  69 

84.26 

73.34 

10.92 

4.33 

0.000032 

55.00 

6.69 

+  00 

DDG  70 

72.83 

68.17 

4.66 

1.41 

0.083385 

45.05 

-0.91 

+  00 

DDG  73 

69.10 

70.28 

-1.18 

-0.38 

0.647678 

47.61 

-6.37 

+  00 

DDG  76 

63.53 

64.15 

-0.63 

-0.16 

0.564648 

39.57 

-7.08 

+  00 

Cluster  analysis  was  again  used  to  discover  the  reason  that  these  10  ships  experienced 
a  statistically  significant  change.  Figure  14  shows  that  during  the  brownout  period,  ships  in 
Cluster  1,  similar  to  the  CILS— TAT  period  results,  had  the  largest  changes  in  source  codes  A, 
C,  G,  and  J.  Flowever,  during  the  brownout  period,  source  codes  G  and  J — versus  codes  A 
and  C — had  a  Type  I  error  less  than  0.05.  Looking  at  the  magnitude  of  the  change  in  means 
among  the  source  codes,  we  see  that  it  is  clear  that  source  code  J  dominated  the  movement  of 
allowance  effectiveness.  For  comparison,  source  code  J’s  ellipse  was  centered  on  a  change 
in  means  of  -1.31  during  the  CILS— TAT  period  and  on  -39.96  during  the  brownout  period; 
source  code  A’s  ellipse  was  centered  on  -57.73  during  the  CILS— TAT  period  and  on  18.90  in 
the  brownout  period.  During  the  brownout  period,  both  the  magnitude  and  statistical 
significance  flipped.  Looking  at  the  parallel  coordinate  plot  at  the  top  of  the  figure,  we  note 
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that  during  the  brownout  period,  the  higher  the  value,  the  better  for  the  change  in  means  of 
source  codes  A,  C,  D,  and  F;  and  the  smaller  the  number,  the  better  for  the  change  in  means 
of  source  codes  G  and  J.  Based  on  this  analysis,  it  is  clear  that  the  change  in  source  code  J  is 
the  reason  that  the  10  ships  experienced  a  statistically  significant  change  in  allowance 
effectiveness. 


Source  Code  Description 


Standardized  Allowance  Effectiveness  and  Variable  Clusters  Means 


A 

Allowance  List  Material  issues 
from  storeroom  stock 

C 

Non-Allowance  List  Material 
(SIM)  issues  from  storeroom  stock 

D 

Allowance  List  Material  Not  In 

Stock  (N1S)  when  requested 

F 

Non-Allowance  List  Material 
(SIM)  Not  In  Stock  (NIS)  when 
requested 

G 

Not  Carried  (NC)  repair  parts 
which  are  not  listed  on  an  APL  in 
the  ship’s  COS  file 

J 

Not  Carried  (NC)  repair  parts 
which  are  listed  on  an  APL.  but 
docs  not  compute  for  an  allowance 

Cluster  Location  &  95%  Confidence  Ellipse 
Cluster  Number: 


P  Value:  Allowance  Effectiveness 


Change  in  Means:  Allowance  Effectiveness* 


Change  in 
Means 


Source  Code  A 
Source  Code  C 
Source  Code  D 
Source  Code  F 
Source  Code  G 
Source  Code  J 


0.009361 

0.139951 

0.378823 

0.014639 

0.111190 

0.251522 

10.26 

5.99 

1.58 

3.05 

3.39 

2.79 

18.09 

34.64 

-16.49 

31.28 

4.13 

29.62 

5.99 

3.09 

-5.52 

3.63 

5.47 

4.10 

-15.83 

4.55 

-13.09 

12.53 

9.92 

12.39 

0.05 

4.92 

-0.39 

3.53 

1.41 

2.61 

-5.80 

5.38 

1.07 

4.32 

3.34 

4.31 

-39.96 

13.83 

-28.17 

18.19 

11.78 

12.01 

0J57258 

0.024020 

0.689032 

0.333200 

0.014703 

0.278194 

0.128745 

0.468553 

0.956611 

0.177577 

0.455850 

0.076607 

0.820998 

0.358858 

0.842876 

0.268588 

0.334407 

0.209272 

0.543827 

0.022968 

0.565930 

0.362566 

0.009545 

0.402406 

0.063805 

0.936188 

0.561863 

0.081574 

0.063764 

0.397449 

0.026774 

0.769961 

0.036699 

0.066252 

0.216851 

0.087970 

P  Value 


Source  Code  A 
Source  Code  C 
Source  Code  D 
Source  Code  F 
Source  Code  G 
Source  Code  J 


Figure  14.  Impact  of  Source  Codes  on  Allowance  Effectiveness  Using  Cluster 

Analysis 

In  conclusion,  it  is  clear  that  the  brownout  period  caused  an  impact  on  a  larger 
number  of  ships  when  compared  to  the  CILS— TAT  period.  However,  looking  at  the  bigger 
picture  and  Figure  15,  which  shows  allowance  effectiveness  and  the  standard  deviation  for 
the  three  different  periods,  it  is  clear  that  the  brownout  period  resulted  in  many  of  the  ships 
returning  to  their  pre-CILS— TAT  means  and  standard  deviations. 
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Figure  15.  During  the  Brownout  Period,  Many  Ships  Return  to  Before  CILS— TAT 

Levels 

In  summary,  five  ships  were  statistically  significant  during  both  the  CILS— TAT 
period  and  the  brownout  period.  Of  those  five  ships,  three  ended  up  with  allowance 
effectiveness  means  that  were  higher  than  their  pre-CILS— TAT  levels,  indicating  a  lasting 
effect  of  the  implementation.  Six  ships  were  not  significant  during  the  CILS— TAT  analysis 
but  returned  to  within  10%  of  their  pre-CILS— TAT  mean  during  the  brownout  phase.  This 
change  is  not  directly  related  to  CILS— TAT,  although  it  is  a  variable  to  consider.  The 
remaining  seven  ships  did  not  see  a  significant  change  in  mean  in  either  of  the  analysis 
windows. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

In  summary,  the  goal  of  this  research  was  to  conduct  a  BCA  of  CILS— TAT  to 
determine  its  historical  effectiveness  against  allowancing  metrics.  The  analysis  examined  the 
before  and  after  periods  to  determine  whether  the  benefits  were  substantial  enough  to  restart 
the  program  following  the  implementation  of  the  Navy’s  ERP.  The  research  identified  three 
specific  questions  used  in  guiding  the  process.  In  the  following  paragraphs,  we  summarize 
our  findings. 

First,  we  asked  the  question  of  how  did  CILS— TAT  impact  allowance  effectiveness 
for  our  sample  and  at  what  cost?  The  findings  suggested  that  CILS— TAT  had  a  direct  and 
positive  impact  on  allowance  effectiveness  for  six  out  of  18  ships.  Of  the  12  remaining  ships, 
nine  saw  a  positive  change  in  allowance  effectiveness,  although  this  change  is  not 
statistically  significant  and  thus  cannot  be  attributed  to  CILS— TAT.  Based  on  these  findings, 
CILS— TAT  does  achieve  the  ends  for  which  it  is  designed.  Regardless  of  whether  the  U.S. 
Navy  re-implements  this  program  or  some  other  targeted  allowancing  technique,  we  have 
found  that  there  is  a  need  throughout  the  fleet  for  this  model. 

Second,  we  explored  the  question  of  whether  there  are  correlations  between 
allowance  effectiveness  rates,  POTF  from  CASREPs,  and  CILS— TAT  cost.  Our  research 
concluded  that  there  was,  in  fact,  no  direct  correlation  between  the  effectiveness  of  CILS— 
TAT  and  these  three  variables.  Of  the  ships  that  were  statistically  significant,  their 
performance  as  related  to  allowance  effectiveness  and  POTF  could  not  be  linked  directly  to 
the  amount  of  funds  obligated. 

Last,  we  sought  to  answer  the  question  of  the  effect  of  not  having  CILS— TAT  during 
2009-2011,  when  traditional  allowancing  procedures  were  not  available.  The  research 
demonstrated  that  in  the  absence  of  targeted  allowancing  techniques,  ships  that  received 
CILS— TAT  were  able  to  maintain  allowance  effectiveness  rates  within  10%  of  their  pre- 
CILS— TAT  averages  for  up  to  24  months  following  implementation.  These  results  are  most 
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encouraging  because  they  display  the  longer  term  impact  of  CILS— TAT,  as  well  as  provide 
an  expected  shelf  life  for  each  iteration  of  the  process. 

Upon  concluding  our  research,  we  determined  that  CILS— TAT  did  directly  contribute 
to  the  improved  allowance  effectiveness  of  ships  within  the  BMD  fleet.  Ships  in  the  sample 
that  saw  increases  in  allowance  effectiveness  were  able  to  maintain  those  higher  levels  of 
performance  for  up  to  two  years  in  both  the  presence  and  absence  of  traditional  allowance 
maintenance  processes.  Although  CILS— TAT  did  not  improve  allowance  effectiveness  to 
the  same  extent  on  every  ship,  the  fact  that  it  did  improve  one  third  of  our  sample  cannot  be 
ignored,  and  we  recommend  that  the  Navy  continue  to  utilize  this  program  to  improve 
allowance  effectiveness  in  the  future. 

1.  Recommendations  for  Change 

How  can  the  effectiveness  of  a  CILS— TAT  product  be  measured?  This  depends  on 
the  priority  of  the  decision-maker  as  to  whether  CILS— TAT  should  be  used  to  primarily 
improve  allowance  effectiveness  with  a  second  order  effect  of  reducing  the  number  of 
CASREPs  or  whether  the  program  should  first  reduce  CASREP  frequency  and  then  focus  on 
allowance  effectiveness  if  resources  exist.  We  believe  that  the  system  could  be  tailored  in 
either  direction. 

CILS— TAT  seeks  to  improve  allowance  effectiveness  first  with  no  consideration 
given  to  the  priority  of  critical  systems.  Allowances  added  as  a  result  of  CILS— TAT  are 
funded  on  an  all-or-nothing  basis,  and  when  funding  is  constrained,  use  of  a  knapsack  model 
would  aid  in  optimizing  dollars  spent.  For  example,  funding  could  be  applied  to  all  9X 
Cognizance  Code  (COG)  items  with  MCCs  2,  3,  or  4  first  because  this  family  of  COGs 
accounts  for  34.76%  of  all  CASREPs  recorded  and  only  3.02%  of  total  CASREP  cost. 
Additionally,  MCCs  2,  3,  and  4  accounted  for  75.17%  of  all  CASREPs  during  the  analysis 
window.  If  more  priority  were  given  to  these  MCCs,  the  same  effect  on  overall  system 
readiness  could  be  realized  at  half  of  the  cost.  Over  the  time  period  analyzed,  CILS— TAT 
requisitions  were  funded  at  the  rates  shown  in  Table  13. 
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Table  13.  Breakdown  of  CILS— TAT  Requisition  Cost  (2003-2009) 


CILS-TAT  Requisition  Cost 


Cognizance  Code  (COG) 


1 

3 

7 

9  Grand  Total 

1 

0.53% 

0.65% 

33.95% 

4.15%  39.27% 

2 

0.06% 

0.31% 

10.52% 

1.94%  12.84% 

MCC  J 

0.10% 

0.21% 

25.05% 

1.88%  27.25% 

4 

0.03% 

0.06% 

2.63% 

0.39%  3.10% 

5 

0.00000% 

0.00000% 

0.00000% 

0.00005%  0.00005% 

N/A 

0.74% 

0.49% 

12.13% 

4.19%  17.54% 

Grand  Total 

1.44% 

1.72% 

84.29% 

12.55%  100.00% 

As  shown  in  Table  13,  nearly  40%  of  all  funding  applied  as  a  result  of  CILS— TAT  is 
used  for  NIINs  with  MCC  1  (failure  results  in  minor  mission  impact).  By  using  a  nearly 
identical  amount  of  funding  across  MCCs  2,  3,  and  4,  a  much  larger  impact  could  be  realized 
on  the  ship’s  ability  to  perfonn  its  mission. 

Figures  16  and  17  clearly  demonstrate  the  effect  of  making  a  change  in  funding 
priorities  by  using  MCCs  as  a  factor.  Results  are  based  on  the  analysis  of  13,956  CASREPs 
submitted  by  sample  ships  during  the  analysis  window. 
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■  9  COG 

0.27% 

0.99% 

1.18% 

0.85% 

0.22% 

■  7  COG 

13.35% 

10.91% 

32.37% 

27.73% 

10.17% 

■  3  COG 

0.09% 

0.50% 

0.36% 

0.31% 

0.11% 

■  1  COG 

0.33% 

0.08% 

0.07% 

0.07% 

0.04% 

Mission  Criticality  Codes  (MCC) 


Figure  17.  Breakdown  of  Casualty  Report  Cost 

2.  Potential  Obstacles 

The  primary  obstacle  to  changing  the  implementation  of  CILS— TAT  would  be  in 
making  the  shift  away  from  allowance  effectiveness  as  a  primary  measure  to  an  approach 
focused  more  on  critical  systems.  In  a  perfect  world,  all  systems  would  operate  as  required  at 
all  times,  but  in  reality,  certain  systems  will  experience  difficulty  and  operate  in  a  degraded 
capacity.  For  this  reason,  it  is  important  that  we  focus  more  closely  on  critical  systems  both 
during  the  initial  allowance  provisioning  process  and  during  times  when  allowances  are 
being  maintained.  By  shifting  to  an  MCC-based  allocation  model,  we  would  be  moving 
away  from  some  of  the  more  traditional  supply  effectiveness  metrics.  Because  this  change 
would  be  both  cultural  and  procedural  in  nature,  it  makes  implementation  the  most  difficult. 

This  change  would  result  in  a  decrease  in  allowance  effectiveness  but  an  increase  in 
the  ability  of  a  ship  to  carry  out  mission  tasking.  In  an  era  where  the  DoD  is  expected  to 
operate  at  prior  years’  levels  with  fewer  resources,  perhaps  this  cultural  change  is  necessary. 


B.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

During  the  course  of  our  research,  we  identified  some  areas  where  our  research  could 
be  improved  or  additional  questions  that  could  be  answered  through  the  use  of  a  larger 
sample  size,  as  well  as  a  more  diverse  set  of  platforms  selected.  An  additional  improvement 
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to  this  research  would  be  to  select  a  control  group,  or  in  this  case,  a  control  ship.  This  ship 
would  be  evaluated  over  the  same  time  period;  however,  it  would  not  receive  a  CILS— TAT. 
Its  performance  could  then  be  compared  against  several  other  ships  of  the  same  class  or 
mission  area  to  determine  the  long-tenn  benefits  of  the  program  at  the  unit  level. 

There  are  other  variables  that  could  also  contribute  to  the  supply  effectiveness  of  a 
ship  that  were  not  evaluated  here.  Particularly,  if  supply  gross  and  net  effectiveness  are  to  be 
evaluated,  then  operational  funding  has  to  be  considered  as  well.  Improved  operational 
funding  levels  will  increase  the  opportunity  for  a  ship  to  replenish  its  stock  and  thereby 
reduce  the  number  of  not-in-stock  (NIS)  demands.  Our  research  evaluated  the  NC  rate  for 
each  ship  and  was  therefore  independent  of  periodic  OPTAR  funding  grants. 

A  study  should  be  conducted  comparing  the  allowance  maintenance  done  through 
traditional  processes  as  well  as  through  CILS— TAT.  We  were  unable  to  gain  access  to  this 
data,  but  we  hypothesize  that  those  ships  in  our  study  that  did  not  see  a  significant  statistical 
change  attributed  directly  to  CILS— TAT  may  have  been  a  result  of  increased  traditional 
allowance  products  (e.g.,  more  allowance  adds  through  ASIs).  The  results  of  this  new  study 
would  provide  a  more  accurate  understanding  as  to  the  precise  impact  of  CILS— TAT. 

Lastly,  in  order  to  grasp  CILS— TAT’ s  impact  on  CASREP  frequency,  additional 
analysis  could  be  conducted  on  NIINs  added  as  a  result  of  CILS— TAT.  Because  the 
allowances  added  to  a  ship  following  a  CILS— TAT  are  derived  from  previously  NC  demands, 
a  percentage  of  those  demands  were  recorded  as  CASREPs.  Analysis  could  be  conducted  to 
determine  the  demand  frequency  of  previous  CASREP  NIINs  that  are  added  to  the  ship’s 
stock  and  subsequently  avoid  a  new  CASREP. 
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